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(54) Information processing system and method therefor 



(57) In an information processing system wherein a 
printer and a personal computer are connected togeth- 
er, when the printer receives a print job from a scanner, 
as there are a plurality of processes corresponding to 
the received printed job, the printer distinguishes be- 
tween a print process to be performed by the printer and 



a print notification process to be performed for a user by 
the personal computer. When the printer initiates print- 
ing, it instructs the personal computer to perform the 
print notification process. Upon receipt of the instruction, 
the personal computer calls a user to notify the user that 
printing has been initiated. 
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Description 

The present invention relates to an information 
processing system to which a plurality of devices are 
connected, and a method therefor. 

In order to execute various processes in an envi- 
ronment wherein a plurality of devices are available, 
usually a user selects a suitable device for each proc- 
ess, and then instructs the selected devices to execute 
the corresponding processes. 

For example, when information prepared by a com- 
puter, or information read by a scanner is to be trans- 
mitted in a printed form to a person at a remote location, 
a printer belonging to the person connected via a net- 
work is designated and printing can be performed by the 
designated printer. 

However, when a user can not send an instruction 
directly to a device that is connected to a network, that 
device can not be instructed to execute the process. 

According to the conventional technique, for a case 
wherein a selected device malfunctions, or does not 
process the required functions, and can not perform a 
process as instructed, or for a case wherein, even 
though the device can perform the process as instruct- 
ed, the original object of the process has not yet been 
achieved, depending on the status, the process as- 
signed to the device can not be performed as is desired 
by the user. To resolve this problem, the user must con- 
firm the results ol the processing and must send an in- 
struction to another device to perform the same process 
or an additional process. 

In the above transmission example, when a person 
is not present at a transmission destination, information 
that is transmitted can not be acquired by that person, 
so that there is a problem will arise concerning the trans- 
mission of an urgent document. To avoid this, whether 
or not a person is present at a destination is determined. 
When the person is not present, the person's current 
location must be acquired and the data must be trans- 
mitted there. Furthermore, to notify the person of the 
transmission, the transmission source must contact the 
person as well as transmitting the information. 

An aspect of the present invention provides an in- 
formation processing system that can reduce the oper- 
ating load placed on a user, and a control method there- 
for. 

Another aspect of the present invention provides an 
information processing system that sends an instruction 
to only one device in order for the performance of proc- 
esses that are to be performed by a plurality of devices, 
and a control method therefor. 

A further aspect of the present invention provides 
an information processing apparatus that can issue an 
instruction to another device in accordance with an in- 
struction required from a user, and a control method 
therefor. 

According to one aspect, the present invention re- 
lates to an information processing system that includes 



a first device and a se~~nd device that is connected to 
the first device, the fir: vice comprising: 

first reception me> >r receiving information. 

5 identification mear^ _ r, as a plurality of proces. 

correspond to the information, distinguishing be- 
tween a first process to be performed by the first 
device and a second process to be performed by 
the second device, 

to first execution means for performing the first proc- 
ess, and 

instruction means for instructing the second device 
to perform the second process; 
and the second device comprising: 
*5 second reception means for receiving an instruction 
Irom the first device, and 

second execution means for performing the second 
process. 

20 According to another aspect, the present invention 
relates to an information processing method, for an in- 
formation processing system that includes a first device 
and a second device that is connected to the first device, 
comprising: 

25 

a reception step of receiving information at the first 
device, 

an identification step of, among a plurality of proc- 
esses correspond 1o the information is received at 

30 the first device, distinguishing between a first proc- 

ess to be performed by the first device and a second 
process to be performed by the second device, 
a first execution step of performing the first process 
at the first device, and 

35 an instruction step of the first device transmitting an 
instruction to perform the second process to the 
second device; and 

a second execution step of the second device per- 
forming the second process upon receipt of the in- 
40 struct ion from the first device. 

According to an additional aspect, the present in- 
vention relates to an information processing apparatus, 
connected to another information processing appara- 
45 tus, comprising: 

reception means for receiving information; 
identification means for, among a plurality of proc- 
esses correspond to the information, distinguishing 
so between a first process to be performed by the in- 
formation processing apparatus and a second proc- 
ess to be performed by the other information 
processing apparatus'; 

execution means for performing the first process; 
55 and 

instruction means for instructing the other informa- 
tion processing apparatus to perform the second 
process. 



2 



3 



EP 0 855 640 A1 



4 



According to a further aspect, the present invention 
relates to an information processing method, for an in- 
formation processing apparatus connected to another 
information processing apparatus, comprising: 

5 

a reception step of receiving information; 
an identification step of, among a plurality of proc- 
esses that correspond to the information, distin- 
guishing a first process to be performed by the in- 
formation processing apparatus and a second proc- io 
ess to be performed by the other information 
processing apparatus; 

an execution step of performing the first process; 
and 

an instruction step of instructing the other informa- 75 
tion processing apparatus to perform the second 
process. 

According to a still further aspect., the present in- 
vention relates to an computer-readable storage medi- 20 
um on which is stored an information processing pro- 
gram for controlling a computer to process information, 
the program comprising codes for causing the computer 
to perform: 

25 

a reception step of receiving information; 
an identification step of, among a plurality of proc- 
esses that correspond to the information, distin- 
guishing between a first process to be performed 
by the information processing apparatus and a sec- 30 
ond process to be performed by the another infor- 
mation processing apparatus; 
an execution step of performing the first process; 
and 

an instruction step of instructing the other informa- 35 
tion processing apparatus to perform the second 
process. 

Embodiments of the present invention will now be 
described with relerence to the accompanying drawings *o 
in which: 

Fig. 1 is a block diagram illustrating the arrange- 
ment of an information processing system accord- 
ing to a first embodiment of the present invention; 45 
Fig. 2 is a diagram illustrating the functional ar- 
rangement according to the first embodiment; 
Fig. 3 is a diagram illustrating the hardware ar- 
rangement of the individual units according to the 
first embodiment; so 
Fig. 4 is a flowchart showing the processing per- 
formed for the first embodiment; 
Fig. 5 is a diagram showing a process for selecting 
an optimal printer from a plurality of printers and 
performing a printing job with it; S5 
Fig. 6 is a diagram for explaining a case where a 
controller is used; 

Fig. 7 is a diagram showing a case where a docu- 



ment is read by a scanner for transmission to a spe- 
cific printer; 

Figs. 8, 9 and 1 0 are diagrams illustrating a case 
where a* document is read by a scanner for trans- 
mission to a specific printer that the scanner can not 
control; 

Fig. 11 is a diagram illustrating a case where infor- 
mation concerning the state of a printer is distribut- 
ed; 

Fig. 12 is a flowchart showing the processing per- 
formed an eighth embodiment; 
Fig. 1 3 is a table containing remarks for determining 
whether or not an advance notice should be trans- 
mitted in consonance with job types and conditions; 
Figs. 14A and 14B are respectively diagrams of a 
case where a user is informed in advance ot infor- 
mation to be filed and a case where unnecessary 
information is abandoned; 

Fig. 15 is a flowchart showing the processing for a 
ninth embodiment; 

Fig. 16A is a diagram showing the transmission of 
information for the ninth embodiment; 
Fig. 16B is a diagram showing the transmission of 
information when an apparatus corresponding to 
the object of a job is not present; 
Fig. 17 is a diagram showing an example display 
for a window by which an optimal method is pro- 
posed to a user; 

Fig. 18 is a flowchart showing the detailed proce- 
dure for job execution; 

Fig. 19 is a diagram showing an example of classi- 
fied information to be read; 
Fig. 20 is an example job table; 
Fig. 21 is a diagram showing an example printing 
window; 

Fig. 22 is an example job table; 

Fig. 23 is a diagram showing the contents of <file 

A>: 

Fig. 24 is a diagram showing an inquiry window; 

Fig. 25 is an example job table; 

Fig. 26 is a diagram showing example scheduling 

data; 

Fig. 27 is a diagram showing an example user que- 
ry; 

Fig. 28 is an example job table; 

Fig. 29 is an example job table; 

Fig. 30 is a diagram showing an example change in 

the status of a printer; 

Fig. 31 is a table showing printer statuses; 

Fig. 32 is an example job table; 

Fig. 33 is a diagram showing a query window; 

Fig. 34 is an example job table; 

Fig. 35 is a diagram showing an example printer 

change status; 

Fig. 36 is a table for displaying planning and action 
rules; 

Fig. 37 is a table for displaying planning and action 
rules; 
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Fig. 38 is a diagram showing actions and premise 
conditions for an object; 

Fig. 39 is a table for example apparatus properties; 
Fig. 40 is a table tor example apparatus statuses; 
Figs. 41 A and 41 B are diagrams showing states s 
where a plurality of apparatuses are connected to 
a network; 

Fig. 42 is a diagram showing a functional arrange- 
ment for performing the processing for a fourteenth 
embodiment; io 
Fig. 43 is a main flowchart showing the processing 
for the fourteenth embodiment; 
Fig. 44 is a flowchart showing the processing for 
controlling other apparatuses; 

Fig. 45 is a flowchart showing the processing per- is 
lormed by one's own apparatus; 
Fig. 46 is a diagram showing an example where a 
printer other than a designated printer is employed 
for output; 

Fig. 47 is a diagram showing an example where a 20 
designated printer transfers a job to another printer; 
Fig. 46 is a diagram showing an example where a 
user's printer transfers a job to a printer at a user's 
destination; 

Fig. 49 is a flowchart showing the processing for a 2s 
sixteenth embodiment; 

Fig. 50 is a conceptual diagram showing the 
processing for the sixteenth embodiment; 
Fig. 51 is a diagram showing transmission of a job 
for a seventeenth embodiment; 30 
Fig. 52 is a flowchart showing the processing per- 
formed by a print job automatic setup unit; 
Fig. 53 is a table showing printing periods and con- 
sumption of toner for a plurality of printers for print- 
ing a single character; 35 
Fig. 54 is a diagram showing the contents of a print 
job and parameters designated by a user; 
Fig. 55 is a flowchart showing the processing per- 
formed by a print job automatic change unit; 
Fig. 56 is a diagram showing the contents of a print 40 
job and parameters designated by a user; 
Fig. 57 is a diagram showing transmission of a job 
for a nineteenth embodiment; 
Fig. 58A is a diagram for explaining an example 
where parameters are set for a print job by employ- 
ing a natural language, 

Fig. 58B is a table showing the meanings of char- 
acter strings in the parameter setup, and request 
items with which to complete the meanings; 
Fig. 59 is a diagram showing an example where pa- so 
rameters of a print job are set by using a document; 
Fig. 60 is a diagram showing transmission of a job 
for a twenty-first embodiment; 
Fig. 61 is a flowchart showing the processing per- 
formed by a print job simulation unit; ss 
Fig. 62 is a flowchart showing the processing per- 
formed by a printer system according to a twenty- 
second embodiment; 



Figs. 63A and 63B are diagrams illustrating the ex- 
ternal appearance of the printer system according 
to the twenty-second embodiment; 
Fig. 64 is a flowchart showing the processing per- 
formed for a twenty -third embodiment; 
Fig. 65 is a diagram showing the functional arrange- 
ment of a system according to the twenty-third em- 
bodiment; 

Fig. 66 is a diagram showing a system for perform- 
ing the overall process according to the twenty-third 
embodiment and input/output types relative to an 
external apparatus; 

Fig. 67 is a diagram showing an example where a 

schedule is extracted from electronic mail; 

Fig. 68 is a flowchart showing specific processing 

performed for the twenty-third embodiment; 

Fig. 69 is a diagram showing an example schedule 

conflict; 

Fig. 70 is a diagram showing an example where a 
user aurally queries a system; 
Fig. 7t is a flowchart showing the processing for a 
twenty-fourth embodiment; 

Fig. 72 is a diagram showing an example rearrange- 
ment of conflicting schedules; 
Fig. 73 is a flowchart showing the processing for re- 
planning conflicting schedules; 
Fig. 74 is a flowchart showing the processing for re- 
planning responses; 

Fig. 75 is a diagram showing an overall image in- 
cluding the input/output of a system according to a 
twenty-sixth embodiment; 

Fig. 76 is a flowchart showing the overall processing 
performed by the system according to the twenty- 
sixth embodiment; 

Fig. 77 is a flowchart showing the processing per- 
formed by an input management unit; 
Fig. 78 is a flowchart showing the processing per- 
formed by a core unit; 

Fig. 79 is a flowchart showing the processing per- 
formed by an output management unit; 
Fig. 80 is a flowchart showing data derivation 
processing; 

Fig. 81 is a diagram showing an example document 
from which data are to be extracted; 
Figs. 82A and 82B are diagrams for explaining a 
specific block reading process; 
Fig. 83 which is composed of Figs. 83A and 83B are 
flowcharts showing determination procedures in a 
process for referring to date data; 
Fig. 84 is a diagram showing transmission of infor- 
mation according to a twenty-seventh embodiment; 
Fig. 85 is a diagram showing transmission of infor- 
mation according to a twenty-eighth embodiment; 
Fig. 86 is a diagram showing transmission of infor- 
mation according to the twenty-eighth embodiment; 
Fig. 87 is a flowchart showing the processing for the 
twenty-eighth embodiment; 

Fig. 88 is a table showing knowledge of a common 
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knowledge base; 

Fig. 89 is a table showing knowledge of a common 
knowledge base in a field specified on a cover page; 
Fig. 90 is a diagram showing an example where 
voice is used to reter to processing history to per- 5 
form a process; 

Fig. 91 is a flowchart showing the processing per- 
formed according to a thirtieth embodiment; 
Fig. 92 is a diagram illustrating an example input 
document; 10 
Fig. 93 is a diagram showing the contents of a con- 
versation between a system and a user; 
Fig. 94 is a diagram showing an output document 
that a system prepares based on the judgement of 
the conversation; 75 
Fig. 95 is a flowchart tor an example where a re- 
quired process is automatically performed after a 
user is queried; 

Fig. 96 is a diagram showing an example wherein 
a system is operated by a controller; 20 
Fig. 97 is a diagram showing an example status 
monitor; 

Fig. 98 is a diagram showing an example wherein 
a controller recognizes a target model and displays 
a corresponding Ul; 25 
Fig. 99 ts a flowchart showing the processing per- 
formed by a controller for the example wherein the 
controller recognizes a target model and displays a 
corresponding Ul; 

Fig. 100 is a flowchart showing the processing per- 30 
formed by a target model for the example wherein 
the controller recognizes a target model and dis- 
plays a corresponding Ul; 

Fig. 101 is a diagram showing an example wherein 

a controller receives a Ul from a target model and 35 

displays it; 

Fig. 102 is a flowchart showing processing per- 
formed by a controller for the example wherein the 
controller receives a Ul from a target model and dis- 
plays it; 40 
Fig. 103 is a flowchart showing processing per- 
formed by a target model for the example wherein 
the controller receives a Ul from the target model 
and displays it; 

Fig. 104 is a diagram showing an example wherein 
a controller receives a Ul from a target model via a 
wireless LAN rather than directly, and displays it; 
Fig. 105 is a flowchart showing the processing per- 
formed by a controller for the example wherein the 
controller receives a Ul via a wireless LAN and dis- so 
plays it; 

Fig. 106 is a flowchart showing the processing per- 
formed by a target model for the example wherein 
the controller receives a Ul via a wireless LAN and 
displays it; 55 
Fig. 107 is a flowchart showing the processing per- 
formed by a server for the example wherein the con- 
troller receives a Ul via a wireless LAN and displays 



it; 

Fig 108 is a diagram showing an example wherein 
via a wireless LAN a controller receives a different 
Ul for each user and displays it; 
Fig. 109 is a flowchart showing the processing per- 
formed by a controller for the example wherein via 
a wireless LAN a controller receives a different Ul 
for each user and displays rt; 
Fig. 110 is a flowchart showing the processing per- 
formed by a target model for the example wherein 
via a wireless LAN a controller receives a different 
Ul for each user and displays it; 
Fig. 111 is a flowchart showing the processing per- 
formed by a server for the example wherein via a 
wireless LAN a controller receives a different Ul for 
each user and displays it; 

Fig. 112 is a table showing a Ul determined in con- 
sonance with a target model and user identification 
information; 

Fig. 11 3 is a diagram illustrating an example where- 
in a model in front that is to be operated performs 
a function that the model does not have; 
Fig. 114 is a diagram illustrating an example where- 
in a model in front that is to be operated performs 
a function that the model does not have; 
Fig. 115 is a diagram showing an example per- 
formed when communication can not be performed 
directly with a model in front that is to be operated; 
Fig. 116 is a flowchart showing the processing per- 
formed by a controller according to a thirty-ninth 
embodiment; 

Fig. 117 is a diagram showing an example wherein 
an operation/action' by a user and a time are jointly 
stored as history along with important associated 
information; 

Fig. 118 is a flowchart for the processing performed 
by a controller according to a fortieth embodiment; 
Fig. 1 1 9 is a flowchart for the processing performed 
by a target model according to the fortieth embodi- 
ment; 

Fig. 1 20 is a flowchart for the processing performed 
by a server according to the fortieth embodiment; 
Fig. 121 is a table showing example updated history 
information; 

Fig. 1 22 is a diagram illustrating an example where- 
in an operation is performed based on history: 
Fig. 123 is a diagram illustrating the example 
wherein the operation is performed based on histo- 
ry; 

Fig. 1 24 is a diagram showing an example structure 
for a controller; 

Fig. 125 is a diagram showing another example 
structure for a controller; and 

Fig. 126 is a diagram showing an additional exam- 
ple structure for a controller. 



5 



9 



EP 0 855 640 A1 



10 



[First Embodiment] 

Fig. 1 is a diagram illustrating a system arrange- 
ment according to a first embodiment in which a PC 
(Personal Computer) 101 , a scanner 102, a color printer s 
103 and a monochrome printer 104 are connected to a 
network. 

In this embodiment; when an apparatus in a system 
wherein a plurality of apparatuses are connected to- 
gether is instructed to execute a specific job, the appa- io 
ratus analyzes the object of the job by examining its con- 
tents, and thus ascertains which job to execute. As a 
result of the analysis, the apparatus acquires necessary 
information concerning the job and thereafter executes 
it. 

It for the execution ot a job there is available a more 
effective method or another and better apparatus than 
an instructed method or an assigned apparatus, an in- 
struction is issued to the better apparatus or a notice is 
transmitted to a user to propose the use ot the more ef- 
fective method. Further, as a consequence of the exam- 
ination of the contents of the job the instruction may be 
rejected. 

Job types are an operation input by a user, an exe- 
cution of an instruction received from another appara- 
tus, the acquisition of results by analyzing a job, or a job 
that is newly generated by an owned apparatus during 
an idling period. When, for example, a document is to 
be read by a scanner and printed by a printer, the fol- 
lowing jobs are generated. 

(1 ) Analysis of the contents of an instruction that is 
transmitted from a user to a scanner and that is re- 
lated to a document set in the scanner; 

(2) A reading and anafyzation process for a docu- 
ment, and an instruction process for instructing a 
printer to print a document that has been read, both 
of which are generated by analyzing an instruction 
from a user that is received by the scanner (and the 
fact that the document has been set in the scanner). 

(3) Generation by the scanner of a printing instruc- 
tion for a printer during the instruction process. 

(4) Generation of a printing process by the printer 
as a result of an analysis ot the printing instruction 
received from the scanner. 

A method for generating and managing these jobs 
will later be described in detail by referring loan eleventh 
and following embodiments. 

The procedures according to which a plurality ot ap- 
paratuses execute the above described processes will 
be specifically described while referring to the drawings. 

Fig. 2 is a diagram illustrating the functional ar- 
rangement for performing the processing in this embod- 
iment. A job reception unit 201 receives a job from a 
user or another apparatus. The received job is regis- 
tered in a job table. A job analysis unit 203 reads and 
analyses a job registered in the job table 202. Another 



apparatus information acquisition unit 204 acquires the 
attributes and the current states of the other apparatus- 
es that are connected. A state determination unit 207 
determines the state from the current states of the other 
apparatuses. 

An optimum planning unit 206 makes an optimal 
plan for the execution of a job. A job execution determi- 
nation unit 207 determines whether a job should be ex- 
ecuted, or when the job is to be executed, and deter- 
mines whether an owned apparatus or another appara- 
tus will execute the job. When another apparatus exe- 
cutes the job, another apparatus instruction unit 208 
sends an instruction to another apparatus for the job ex- 
ecution. When the owned apparatus executes the job, 
a job execution unit 209 executes the job An execution 
notification unit 210 notifies a user that the execution 
instruction has been issued to another apparatus. 

Fig. 3 is a diagram showing the hardware arrange- 
ment of the units that perform the processing in this em- 
bodiment. 

An I/O 301 exchanges input/output data with exter- 
nal apparatuses. A CPU 302 executes a program and 
controls the individual units. A ROM 303 is used to store 
programs, which correspond to individual flowcharts 
that will be described later and which are to be executed 
by the CPU 302, and fixed data. A RAM 304 is used to 
temporarily store variables and intermediate data, such 
as the attributes and the current states of the other ap- 
paratuses acquired by the other apparatus information 
acquisition unit 204, that are generated during the 
processing. A program from an external source may be 
loaded into the RAM 304 and stored therein. 

Fig. 4 is a flowchart showing the processing per- 
formed for the first embodiment. 

At step S109, the contents of the job table are ini- 
tialized. At step S110, in order to determine whether a 
job has been input, a check is performed to determine 
whether or not a job has been input by a user, a job has 
been input by a device that can be detected by the sys- 
tem, from results obtained by analyzing the job, or a new 
job has been generated by an owned apparatus during 
idling. When a job has been input, at step S111 , the input 
job is added to the job table. At step S112, the job table 
is examined to determine whether there is job. If there 
is a job, at step S11 3 the job is extracted. At step S114 
a received instruction is analyzed, and from the infor- 
mation thus obtained, the object of the job is acquired. 
At step S115, the analyzed instruction is employed to 
search for a job that is to be executed. At step S116 a 
job that is searched for at S115 is registered. 

At step S117 a check is performed to determine 
whether or not there is a job that, has not yet been exe- 
cuted. If there is such a job, program control advances 
to step S118. At step S118 a check is performed to de- 
termine whether or not the job can be executed. When 
it is ascertained that the job can not be executed, pro- 
gram control returns to step S115 and a job that must 
be executed is searched for. At step S119 the job is ex- 



25 



30 



35 



40 



45 



50 



11 



EP 0 855 640 A1 



12 



ecuted and program control returns thereafter to step 
*S117. 

If a user sends an instruction, for example, to read 
a document using a scanner and to print the document 
using a printer it is assumed that, as a result of the anal- 
ysis, the acquisition of a printed document is the object 
of a user, in this case, a document reading and analysis 
process, which is generated by analyzing information 
received by the scanner from the user, and a process 
for instructing a printer to print the document that has 
been read are searched for as jobs. These processes 
will be described in detail during the course of the elev- 
enth and the following embodiments while referring to 
specific examples. 

Fig. 5 is a diagram showing a process for selecting 
from among a plurality of printers an optimal printer tor 
performing printing. 

In Fig. 5, job information is input/transmitted as a 
command from a host computer or an instruction from 
a remote controller, or is obtained by analyzing a voice 
command issued by a user using natural language, or 
from bar code or a sentence read from a cover page. 

A device that has received the job inlormation ana- 
lyzes the job, and then selects a printer that can execute 
a process suitable for the job and outputs the data to the 
printer. Further, when there is an instruction for notifica- 
tion, the device selects an optimal medium and dis- 
patches a notice thereto. 

Therefore, upon the receipt of a request from a user, 
the structure of the printer is referred to, and an appro- 
priate printer is selected to perform the printing. At this 
time, in consonance with the situation, the processing 
can be allocated for a plurality of printers. Further, when 
there is an instruction for notification, an optimal medium 
is selected to dispatch a notification that the printing has 
been terminated along with the name of the printer that 
has performed the printing. 

A request submitted by a user concerns paper siz- 
es, printing quality, colors and time limit (e.g., comple- 
tion of printing by 5 o'clock). The printer configuration 
covers the printer types and the number of printers in 
the system, and their performances or their current sta- 
tuses (whether the printers are normally operated, 
whether there is a printing queue, or whether paper or 
toner is insufficient). As for a medium and a method 
used for notification, a notification is transmitted by 
voice through a user's terminal or by telephone, or a no- 
tification message is transmitted to a pager, or a docu- 
ment describing the contents of a notification is trans- 
mitted by electronic mail or by facsimile. 

A situation where the individual units perform the 
processing shown in Fig. 5 will now be described while 
referring to the flowchart in Fig. 4. 

First, if, at step S110, an urgent instruction for out- 
putting a specific document by five o'clock is transmitted 
to a host computer, at step S111 a job is added to a job 
table. At step S1 1 3, the job is extracted from the job table 
and at step S114 the received instruction is analyzed to 



obtain the object of the job. 

At step S115, the following jobs are searched for 
following the analysis of the instruction. 

s ( i ) Acquisition of specifications (paper size, printing 

quality, color document, etc. ) to be used for select- 
ing a suitable printer for outputting a document, and 
of other information, such as time limit. 

(2) Selection from among the connected printers of 
io a suitable printer for specifications (1). 

(3) Examination of the selected printer to determine 
whether or not any obstacle to printing output exists. 

(4) Output data to a selected printer. 

(5) Issuance of the selected printer and a notifica- 
is tion that the data has been output. 

At step Si 16, the job found at step S115 is regis- 
tered. The above process is repeated until, at step S11 7, 
there are no jobs to be searched for. At step S118 a 
20 check is instituted to determine whether or not the job 
can be performed. When the performance of the job is 
not possible, another job that must be executed is 
searched for. At S1 1 9, the jobs that have been searched 
for are sequentially performed. 

2S 

[Second Embodiment] 

Fig. 6 is a diagram for explaining an example where- 
in a remote controller (hereinafter referred to merely as 

30 a controller) is employed as a means for sending instruc- 
tions to various devices. 

A controller 605, which has a display, displays a 
control panel for, or information concerning a device, 
such as a printer 601 or 602, a facsimile machine 603 

3S or a PC 604, only by approaching the device, and sends 
instructions to the device to control it. Further, the con- 
troller 605 can control a remote device across a network 
via the device that is near the controller 605. 

The controller 605 can also monitor and display the 

40 status of each device, and can acquire the status of a 
remote device on the network via a device that is near 
the controller 605. 

[Third Embodiment] 

45 

Fig. 7 is a diagram showing an example where a 
document is read by a scanner and is transmitted to a 
specific printer. 

When a document has been read by a scanner 701 , 
so a user designates an addressee by voice, etc., or a des- 
tination transmission is designated on a cover sheet, 
and is thus automatically determined. When a mono- 
chrome printer 702 is determined as an addressee, 

ss (1) if the addressee is not in the ready for printing 

state, the document data is transmitted to another 
printer (e.g. , a monochrome printer 703) for printing; 
(2) if a printer designated as an addressee can not 



7 



13 



EP 0 855 640 A1 



14 



print the document because of the conditions spec- 
ified, such as color and paper size, data lor only for 
the pertinent page is transmitted to another printer 
(e.g., a color printer 704 if color printing is request- 
ed); 

(3) data for a document having a large number of 
pages are allocated to a plurality of printers to print; 
or 

(4) if a transfer destination is designated at the ad- 
dressee, the data is transferred to the transfer des- 
tination. 

[Fourth Embodiment] 

Fig. 8 is a diagram showing an example where a 
document is read by a scanner and is transmitted to a 
specific printer that the scanner can not manage be- 
cause it is in a different network domain, etc. 

When, a document is read by a scanner 701 in Aus- 
tin c user designates an addressee by voice, etc., or an 
addressee that is described using a character string or 
a bar code on a cover sheet is read and recognized/ 
interpreted to determine the addressee. As a result, a 
monochrome printer 702 in Tokyo is determined to be 
the addressee. After the document data are transmitted 
to the monochrome printer 702: 

(1) when the monochrome printer 702 is not ready 
for printing, it transmits the data to another printer 
(a monochrome printer 703) for printing; 

(2) if the monochrome printer 702 is not ready for 
printing the received document data because of 
specified conditions, such as color and paper size, 
it transmits either all the data for the received doc- 
ument or only data for pertinent pages to another 
printer (e.g., a color printer 704, if color printing is 
requested); 

(3) if a document has a large number of pages, the 
monochrome printer 702 prints part of the data and 
also allocates printing data to a plurality of other 
printers; or 

(4) if a transfer destination is designated at the mon- 
ochrome printer 702, the printer 702 transfers the 
data to the transfer destination. 

A difference between the third and the fourth em- 
bodiments is that in the third embodiment the scanner 
701 determines the addressee, and in the fourth embod- 
iment the printer 702 that received the document from 
the scanner 701 determines the transfer destination. 

[Fifth Embodiment] 

Fig. 9 is a diagram showing an example wherein a 
document is read by a scanner and is transmitted to a 
specific printer that the scanner can not manage be- 
cause it is in a different network domain, etc. 

When, a document is read by a scanner 701 in Aus- 



tin, a user designates an addressee by voice or using a 
keyboard, or an addressee that is described using a 
character string or a bar code on a cover sheet is read 
and recognized/interpreted to determine the addressee. 

5 As a result, a monochrome printer 702 in Tokyo is de- 
termined as the addressee. After document data are 
transmitted to the monochrome printer 702, they are fur- 
ther transferred to a personal computer 705 that can 
perform a higher level operation. 

to As a result, the personal computer 705 determines, 
from the entry "urgent" on the cover letter, for example, 
that the document is an urgent document. The personal 
computer 705 examines schedule data to find the loca- 
tion of the recipient of the document, and transfers the 

1$ data to that location, for example, Atsugi. At this time, if 
the transfer destination is not a printer but is a different 
medium, such as a facsimile machine 706, medium con- 
version is performed before transmission. 

20 [Sixth Embodiment] 

Fig. 10 is a diagram showing an example wherein 
a document is read by a scanner and is transmitted to 
a specific printer that the scanner can not manage be- 

2S cause it is in a different network domain, etc. 

When, a document is read by a scanner 701 in Aus- 
tin, a user designates an addressee by voice, etc., or an 
addressee that is described using a character string or 
a bar code on a cover sheet is read and recognized/ 

30 interpreted to determine the addressee. As a result, a 
monochrome printer 702 in Tokyo is determined as the 
addressee. After document data are transmitted to the 
monochrome printer 702, they are further transferred to 
a personal computer 705 that can perform a higher level 

35 operation. As a result, the personal computer 705 iden- 
tifies the recipient, prepares a notification sentence, and 
notifies the recipient of the arrival of the document. 

[Seventh Embodiment] 

40 

Fig. 11 is a diagram showing an example wherein 
a notification concerning the status of a printer is trans- 
mitted. 

When a PC 1102 acquires the status of a printer 
45 11 03 across a network and detects a change in the sta- 
tus, such as the occurrence of an error or an event, the 
PC 1102 determines a notification destination, such as 
a user or a manager, in consonance with the contents 
of the change, and employs a medium corresponding to 
50 the determined destination to transmit a notification. For 
example, to transmit a notification to a terminal 1101 of 
a user or a manager, the PC 1102 uses electronic mail, 
or a telephone 1104, or transmits a message to a pager 
1105. 

55 Therefore, in the example in Fig. 10, for example, a 

notification of the reception of the document may be is- 
sued using electronic mail or a pager, instead of the tel- 
. ephone 707. 
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[Eighth Embodiment] 

In this embodiment, before executing an instructed 
job an apparatus notifies a user of the contents of the 
job in accordance with the type or the condition of the 
job, or cancels a job that is determined to be unneces- 
sary for a user. 

Fig. 1 2 is a flowchart showing the processing in this 
embodiment. Fig. 13 is a table showing, in consonance 
with the types and conditions of jobs, whether or not an 
advance notice should be issued. 

in Fig. 12, first, at step S120 a check is performed 
to determine whether or not a job constitutes necessary 
information for a user. If this is true, the processes at 
step S1 22 and the following steps are performed. If not, 
at step S127 the job is canceled. At step S1 22, informa- 
tion concerning the advance notice that corresponds to 
the job type and conditions in Fig. 13 is referred to, or 
the contents described on a cover sheet are analyzed 
to determine whether or not a notice for the process to 
be executed and the contents of the process should be 
issued to a user prior to the performance of the instruct- 
ed job. When advance notice is not required, program 
control moves to step SI 26. When advance notice is 
issued, program control moves to step S124, whereat a 
user is notified of the process to be performed and its 
contents. At step S125, the permission for the perform- 
ance is received from a user, if necessary. At step S126 
the job is performed. 

An explanation will be given for an example where- 
in, when the contents of a job call for the filing of infor- 
mation, an analysis of the system contents is performed 
prior to the filing of information, and information con- 
cerning the contents is transmitted to a user. 

Fig. 14A is a diagram showing an example wherein 
filing process information to be filed is transmitted to a 
user in advance. A file storage unit 1401, a scanner 

1 402 and a user terminal 1403 are provided in this sys- 
tem. The processing performed by the scanner 1402 in 
Fig. 1 4A will now be described while referring to the flow- 
chart in Fig. 12. 

At step St 20, since an instructed job is for informa- 
tion filing, it is assumed that this is necessary for a user. 
At step SI 22, while information designated in Fig. 13 is 
referred to, it is ascertained that a notice concerning in- 
formation to be filed should be issued before the filing, 
and that the information should then be scanned. Pro- 
gram control then moves to step S 124 to issue a notice 
concerning information to be filed. At step S124 the 
scanned information is transmitted to the user terminal 

1403 via a network. At step S126, the filing job is per- 
formed. 

Next, an explanation will be given for an example 
wherein the information on the first page is received and 
analyzed, and when the information is not at all related 
to a user, the following information is not received. 

Fig. 14B is a diagram showing an example wherein 
whether or not a large amount of data to be received is 



required by a user is determined by examining the first 
page, and the receipt of information for the following 
pages is canceled. A reception PC 1404 and a trans- 
mission PC 1405 are provided. 
s The processing performed by the PC 1404 in Fig. 

. 14B will be explained while referring to the flowchart in 
Fig. 12. 

At step SI 20, the first page of the received data is 
analyzed. Since "For All Persons In Charge Of General 
10 Affairs Information" is entered on the first page, it is ap- 
parent that this document is not intended for a user who 
is not in charge of general affairs information, and the 
receipt of the information is determined to be unneces- 
sary. Program control therefore moves to step SI 27. At 
step SI 27, the first page, which has been received, is 
canceled, and information for the second and the follow- 
ing pages is not received. The processing is thereafter 
terminated. 

20 [Ninth Embodiment] 

Fig. 15 is a flowchart showing the processing for a 
ninth embodiment. In this embodiment, an apparatus 
that is designated to perform the processing determines 

25 whether it or another apparatus should perform the 
processing, and as a result of the determination, either 
performs the processing itself or sends an instruction to 
another apparatus to perform the processing. 

At step S150 a check is performed to determine 

30 whether or not a job has been input. If a job has been 
input, at step SI 51 the input job is entered in a job table. 
At step S152 the job table is examined to determine 
whether any jobs have been entered. If they have been, 
at step S153 a job is extracted from the table. At step 

35 S154 a received instruction is analyzed and the object 
of the job is apprehended using the result. At step S155 
the status of the apparatus is examined. At step S156 
the status of another apparatus consonant with the ob- 
ject of the job is examined. 

40 When, as the result of a comparison of the statues 
of the locally owned apparatus and other apparatuses, 
it is found that the locally owned apparatus is optimal for 
the performance of the job, program control moves from 
step S157 to step S158, whereat it is determined that 

45 the owned apparatus will perform the job, and at step 
SI 59 the job is performed by the locally owned appara- 
tus. If there is an optimal apparatus other than the locally 
owned apparatus that has capabilities consonant with 
the object of the job, program control goes from step 

50 S1 60 to step S 1 61 , whereat it is determined that the oth- 
er apparatus will perform the job, and at step S162 the 
locally owned apparatus sends a job execution instruc- 
tion to the apparatus having capabilities consonant with 
the object of the job. At step S1 63 the locally owned ap- 

55 paratus notifies a user that another apparatus has exe- 
cuted the job. 

If there is no apparatus having capabilities conso- 
nant with the object of the job, program control moves 
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from step S1 60 lo step S 1 64, whereat a plan is prepared 
for the use of an optimal method that does not depart 
from the object of the job, and at step S165 the plan is 
proposed to a user. 

Fig. 16A is a diagram showing the transmission of 
information in this embodiment. 

When a user of a PC 101 provides for a printer 1 04 
a job for the output of information input by a scanner 
102, and when the printer 104 has, for example, a print- 
ing malfunction , the printer ascertains that it can not per- 
form the instructed job, and by communicating with oth- 
er apparatuses, it finds a printer 103 that can perform 
the instructed job Thus, the printer 104 determines that 
the printer 1 03 can act for it and perform the job, and so 
instructs the printer 103. Further, the printer 104 trans- 
mits a notification to the PC 1 01 to inform the user of the 
action that it took. In Fig. 16A, the process flow for these 
activities is described by arrows. 

The processes performed by the individual units in 
Fig. 16A will be described while referring lo the flowchart 
in Fig. 15. 

First, the process performed by the scanner 102 will 
be explained. At step S150 the scanner 102 scans a 
document and acquires a job, information concerning 
which is to be transmitted to the printer 104. At step 
S151 this job is entered in the job table, and at step S153 
the job is extracted. At step S154, the scanner 102 
scans additional data and determines that the data is a 
job to be transmitted At step S1 55, the scanner 1 02 as- 
certains that it is operating normally and that no problem 
exists in the scanning and transmission of instructed in- 
formation. 

At step S156 it is ascertained that the job can not 
be transmittedto another apparatus. At step S1 57, since 
optimally the scanner 102 should execute the job, pro- 
gram control moves to step SI 58. At step S158 the 
scanner 102 determines that it should scan the informa- 
tion, and at step S159 the scanner 102 scans the infor- 
mation and transmits it to the printer 104 via a route 1 
on a network. 

At step S150, the printer 104 receives the job for 
the output of received information. At step S151, the 
printer 104 enters the job in the job table, and at step 
S153, it extracts ft. At step S154, the printer 104 ascer- 
tains from the information it receives that the information 
for the user from the PC 101 is to be printed on paper. 
At step S155 the printer 104 determines that it is out of 
toner and disables printing. At step S116 the printer 104 
searches, via the network, for printers that can output 
the received information, and finds the printer 103 (route 
2). 

Since, at step S157, the printer 104 can not execute 
the job, program control moves to step S160. Then, as 
at step S160 it is determined that the job for the output 
of information can be performed by the printer 103, at 
step S161 it is determined that the received information 
should be transmitted to the printer 103. At step S162 
the printer 104 sends an instruction to the printer 103 to 



print the information that is to be received (route 3). At 
step S 163 to notify the use" "at the instructed job was 
performed by the printer 1C sctronic mail for the user 
is transmitted to the PC 10; oute 4). 

s Fig. 16B is a diagram showing the transmission of 

information for a case where no apparatus having ca- 
pabilities consonant with the object of a job is available. 

When the user of the PC 101 provides a job for the 
output to the printer 104 of information input by the scan- 

to ner 102, and when the printer 104 disables printing be- 
cause it is out of toner, the printer 104 communicates 
with other apparatuses, finds the printer 103, which is 
suitable for the object of the job, and determines its sta- 
tus. Then, since the printer 103 disables printing be- 

15 cause it is out of paper, and as there is no other printer 
available that is suitable tor the object of the job, a notice 
is issued to the PC 101 to propose to the user an optimal 
plan whereby when either the printer 103 or 104 has re- 
covered to the printing enabled state, that printer will 

20 perform the job. In Fig. 16B, the process flow for these 
actions is indicated by arrows. 

The procedures performed by the individual units in 
Fig. 1 6B will be explained while referring lo the flowchart 
in Fig. 15. 

25 The process whereby the scanner 1 02 reads a doc- 

ument and transmits it to the printer 104 is performed in 
the same manner as was described previously. 

At step S150 the printer 104 receives a job for the 
output of received information. At step S151, the job is 

30 entered in the job table, and at step S1 53, it is extracted. 
At step S154, the printer 104 ascertains from the infor- 
mation it receives that the information for the user from 
the PC 101 is to be printed on paper. At step S155, the 
printer 10 ascertains that it is out of toner and printing 

3S is disabled. At step S1 56, the printer 104 communicates 
across the network with other printers to find one that 
can print the received information. The printer 104 de- 
termines that the printer 103 has also disabled printing 
because it is out of paper, and that no other appropriate 

40 apparatus is available. 

Since, at step S157, the printer 104 disables the 
performance of the job, program control moves to step 
S160. At step S160, as there is no apparatus that can 
perform the job for the output of information, program 
control goes to step SI 64. At step SI 64, planning is be- 
gun to prepare an optimal method for performing the 
printing, which is the object of the job for the output of 
information. As a result, it is determined that the optimal 
method involves the return of a printing apparatus to its 

so normal operating status. At step SI 65, the proposed 
method, whereby either the printer 104, which is out of 
toner, or the printer 103, which is out of paper, is to be 
recovered to the printing enabled state, is presented in 
a window shown in Fig. 17 for approval by the user. 

55 Then, a response received from the user, or a 

change in the printer status is identified as an input job, 
and the same processing is again performed. In this 
fashion, the object of the user can be achieved. 
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The above described planning will be described lat- 
er in detail during the course of the explanation of spe- 
cific examples for an eleventh and subsequent embod- 
iments. 

[Tenth Embodiment) 

In the processing for the ninth embodiment, where- 
in, at steps S158 and S159, a locally owned apparatus 
determines to perform a job itself, and performs the job, 
before another instructed job is performed, the possibil- 
ity of the occurrence of a problem is analyzed in detail, 
as in the procedures in Fig. 17. When it is ascertained, 
in consonance with environmental conditions or the sta- 
tus of the job performance, that performing the job is not 
advisable, a notice concerning the problem may be is- 
sued, or the request for the performance of the job may 
be rejected. 

When, for example, an instruction for printing a se- 
cret document is issued and a user is not identified as 
an authorized user, the instruction is rejected. 

Fig. 1B is a flowchart showing the detailed proce- 
dures for the performance of a job. 

At step S180 a check is performed to determine 
whether or not a problem exists concerning the perform- 
ance of the instructed job. At step S181 no problem af- 
fecting the performance of a job is found, at step S185 
the job is performed. If a problem is found, at step S182 
a check is performed to determine whether a notice con- 
cerning the problem that was discovered should be 
transmitted to a user, or whether the request for the per- 
formance of the job should be rejected. When it is as- 
certained that such a notice should be transmitted to a 
user, at step S1 84 the notice is transmitted to the user. 
When it is determined that the request for the perform- 
ance of the job is to be rejected, at step S1 86 the request 
for the performance of the job is rejected and a notice 
of rejection is transmitted to the user. 

An example wherein a user of the PC 101 issues 
an instruction to output to the printer 104 confidential 
information read by the scanner 102 will now be de- 
scribed while referring to Figs. 15 and 18. Fig. 19 is a 
diagram showing example confidential information to be 
read. 

First, at slep S150 the scanner 102 scans available 
information and receives a job for the transmission of 
information to the printer 104. At step S151 the job is 
entered in the job table, and at step S1 53, the job is ex- 
tracted therefrom. At step S154 data are scanned, and 
it is ascertained the data constitute a job to be transmit- 
ted. At step SI 55, the scanner 102 determines that it is 
operating normally and that there is no problem affecting 
the scanning and transmission of the instructed infor- 
mation. 

At step S155 it is ascertained that the job can not 
be transmitted to another apparatus. Since, at step 
S157, it is ascertained that optimally the scanner 102 
should perform the job itself, program control moves to 



step S158, whereat the scanner 102 determines that it 
should scan the information itself. 

At step SI 80, as a result of the scanning of the in- 
formation, it is determined to be confidential information 

s whose output is inhibited. Since a problem affecting the 
performance of the job exists, at step S182 it is deter- 
mined that the job (the scanning and the outputttng of 
information) should be rejected. At step S183 the job is 
rejected, and program control thereafter goes to step 

10 S186. At step S186, a notice is issued that the job was 
rejected. 

Similarly, when a plurality of users employ the same 
system, the current user can be identified by a log-in 
name, and an instruction for the output of the contents 

is of files belonging to other users can be rejected. When, 
instead of an instruction from a user, a document is re- 
ceived from the outside and the destination user differs 
from the current user, the output of the received docu- 
ment and the notice of the receipt of the document are 

20 rejected, or the processing is halted until the destination 
user begins to employ the system. 

[Eleventh Embodiment] 

25 The processing for an eleventh embodiment will 
now be described while referring to Fig. 4. At step S1 09 
the job table is initialized. Then, at step S110 a check is 
performed to determine whether there is input from a 
user or from an apparatus that can be detected by the 

30 system. At step S111 an input analysis job is entered in 
the job table to analyze the contents input at step S110. 
At step S 1 1 2 a check is performed to determine whether 
there is a job to be performed. If there is such a job, it 
can be acquired at step S11 3. At step S114 : the object 

35 comprising the background for the performance of the 
job is understood. 

At step S11 5 a possible process is planned by using 
the contents that are input, the status of a device that 
can be detected by the system, and the knowledge that 

40 the system has currently acquired. At step S116 a new 
job is added if necessary. At steps S117 to S119 the 
process as planned at step S115 is performed in conso- 
nance with the contents. The processing is repeated un- 
til no more jobs remain, and program control thereafter 

45 returns to step S1 10. 

A case where a user is to print <file A> will now be 
explained. Fig. 23 is a diagram showing the contents of 
<file A>, the colored picture of a car. Fig. 1 is a diagram 
showing the structure of the system in this embodiment. 

so a user sends an instruction from a PC 1 01 , and a mon- 
ochrome printer 104 is set as a normally used printer. A 
color printer 103 is also set. 

At step S109 the job table is initialized. At step S111 
a job that is to be performed when there is no input is 

55 added to the job table (Fig. 20). When file name <f ile A> 
is designated in a window in Fig. 21 and printing for it is 
selected, at step S1 1 0 it is determined that there is input, 
and program control advances to step S111. At step 
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S111, as is shown in Fig. 22, an input analysis job is 
entered in the job table. Since, at step S112, there is a 
job to be executed, program control advances to step 
S113. At step S113 an analysis of the job to be per- 
formed (input "print <file A>") is acquired. At step S114 
it is ascertained that the object of the input "print <fi!e 
A>" is the printing of the <tile A>. 

At step S115 planning is performed, based on the 
following conditions and statuses, for proposing, to a us- 
er that the printing be performed by a color printer 

* <file A> includes a colored portion (the portion com- 
prising the car in Fig. 23) 

* the color printer 103 that can be used is present 
(Fig. 1). 

At step S1 19 a query "Perform printing by a color 
printer?" is directed to a user (Fig. 24). At the same time, 
a job corresponding to an event is added to the job table 
when no response is received from the user (Fig. 25). 
Program control thereafter returns to step S110. 

When, at step S1 10, there is no input for ten min- 
utes, at step S112 it is assumed that there is a job to be 
performed, and program control advances to step S11 3. 
At step S11 3, a job to be performed (when no response 
is received from a user) is acquired. 

At step S114 it is ascertained that the object for a 
job when there is no response from a user is to deter- 
mine whether a proposal can be accepted. At step S1 1 5 
it is determined from the flow condition and the status 
that the user should be queried by telephone. 

* The telephone number of the current location of the 
user is acquired (schedule data in Fig. 26) 

* The printing priority is urgent (according to the 
schedule data in Fig. 26, file A is to be used at the 
business talk at 13:00). 

At step S11 9 a query, 'Perform printing with a color 
printer?" is directed to the user (Fig. 27). At the same 
time, a job corresponding to an event is added to the job 
table when no response is received from the user (Fig. 
28). Program control thereafter returns to step S110. 

When, at step S110, a user responds with "Yes, 
please" as is shown in Fig. 27, it is assumed that data 
input is to be performed, and program control moves to 
step S111. At step S 1 11 an input analysis job is added 
to the job table (Fig. 21 ). At step S1 12 it is assumed that 
there is a job to be performed, and program control 
moves to step S1 1 3. At step S1 1 3, a job to be performed, 
the input, "Yes, please" is analyzed. At step S1 14 it is 
ascertained that the object of the input, "Yes, please" is 
the acceptance of the proposal. 

At step S115 it is determined that <ftle A> is to be 
transmitted to and printed by the color printer 103. At 
step S118 <file A> is transmitted to and printed by the 
color printer 103. At the same time, a job for confirming 
whether or not the printing has been terminated normal- 



ly is added to the job table (Fig. 29). Program control 
thereafter returns to step Si 10. 

If, at step S110 there is no data input and at step 
S112, as is shown in Fig. 30, the printer status is 

s changed from "Printing" to "Printed document in tray 14 it 
is assumed that there is a job to be executed, and pro- 
gram control advances to step S113. At step S113 a 
process corresponding to the job to be executed, printer 
status, is acquired. At step S114, from a printer status 

io entry in Fig. 31, the status "Printed document in tray" 
means that there is a printed document in a printer tray, 
and the object of the job is understood to be the delivery 
of the printed document to a user. 

At step S115, a waiting state is specified because 

1$ the user may soon come to pick up the printed docu- 
ment. At step S1 1 6, a job for confirming whether the user 
has come and has picked up the printed document is 
added to the job table (Fig. 32). Program control there- 
after returns to step S110. If, at step S110, there is no 

20 input for ten minutes and at step S11 2 the printed doc- 
ument is still on tray, there is a job to be executed; and 
program control therefore advances to step S113. At 
step S113 a process corresponding to the job to be ex- 
ecuted, when the printed document is still on hand, is 

2S acquired. At step S1 14, from the table in Fig. 31 the sta- 
tus "Printed document on tray" means that a printed doc- 
ument is still in the printer tray, and it is assumed that 
the object is the delivery of the printed document to the 
user. Since at step S115 the printed document has not 

30 been picked up for ten minutes, it is assumed that the 
user has forgotten to collect the document, and it is de- 
termined that a notice, "Printed document is in color 
printer tray" is to be issued (Fig. 33). Also, a job corre- 
sponding to the failure of the user to respond and a job 

35 corresponding to the collection of the printed document 
by the user are added to the job table (Fig. 34). Program 
control thereafter returns to step S110. 

When at step S110 there is no input and at step 
S112 the printer status is changed from "Printed docu- 

40 ment in tray" to "Normal" as is shown in Fig. 28, there 
is a job to be executed and program control advances 
to step S113. At step S113 a job corresponding to the 
job to be executed, printed document in tray is re-eval- 
uated. At step S114 the object for the rechecking of the 

45 job corresponding to the "Printed document in tray" is 
understood to be the deletion of an unnecessary job. At 
step S115, it is determined that the job of waiting for a 
response from the user should be deleted, and at step 
S119 the job of waiting for the user's response is delet- 

50 ed. A new job is not added. While the job table is in its 
initial state, program control returns to step S110. 

Figs. 36 and 37 are tables showing plans, actions 
and rules. 

When a user presses the button "Print" in the print- 
ss jng window in Fig. 21 , it is determined by referring to the 
plans and actions in Fig. 36 that the object of the user 
is the changing of the current state to the "Printed doc- 
ument in tray". In addition, since "Printed document in 
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tray" is the assumption for the object "Acquire printed 
document" the final object of the user is determined to 
be "Acquire printed document". 

When a user expresses agreement or disagree- 
ment by voice or by the manipulation of the mouse in s 
response to the query in the inquiry window in Fig. 24 
or the window in Fig. 33, or the telephone query in Fig. 
27, it is determined that the object of the user is the state- 
ment of the agreement or disagreement relative to the 
query. 10 

In Fig. 37 the final object of the system is the thus 
constitutes the achievement of the object of the user. As 
the basis for this, the system must be stable. In addition, 
the system must (1 ) apprehend the object of the user 
and (2) must make a plan for the achievement of the is 
object and execute it. 

To stabilize the system, not only an abnormal timing 
for the system must be adjusted to the normal condition, 
but also the system must be powered off when it is not 
required in order to reduce costs and to prevent the sys- zo 
tern from becoming unstable. 

Fig. 38 is a diagram that for easier understanding 
shows only one part of the table in Fig. 37. 

In order to understand the object of the user, data 
that is input is identified. Further, corresponding to "ac- 2S 
quire printed document" which is the object of the user 
in Fig. 36, "Transfer printed document to user" which is 
the object of the system. 

The presence of a printed document is the premise 
for the transfer of the printed document to a user, and 30 
the object is achieved by notifying the user of the loca- 
tion of the document. Of course, the user must collect 
the document or the document must be delivered to the 
user by some means in order for the user to acquire the 
document; in this embodiment, it is determined that 3S 
when the printed document is no longer in the printer 
tray, that is the equivalent of the user having acquired 
the document. 

The presence of an original document and the print- 
ing conditions are required in order to obtain a printed *o 
document, and it is premised that a corresponding print- 
ing environment normally exists. In this situation, a cor- 
responding printing environment is selected to perform 
printing. The original document must be so prepared 
that its existence is constant. In addition, to clarify the *s 
printing conditions, only when there is a question con- 
cerning a condition need ask a user queries. When a 
printing environment is abnormal, the printing environ- 
ment is normalized to maintain normal environment. 
When no response is received from a user within a pre- so 
determined period of time, additional planning must be 
performed to ascertain the intent of the user. 

The properties of each device are stored in a mem- 
ory in each device, or in the memory of the server that 
manages each device. Fig. 39 is a table showing the ss 
properties of the devices. The statuses of the devices 
are held in the devices, and are voluntarily transmitted 
to other apparatuses by the devices, or by the passive 



transfer of them when requested by another server. 

In this system, the properties are inherent to the in- 
dividual devices and do not change; they are acquired 
when the system is activated or when a connection 
across the network is altered. In the planning process, 
the statuses are acquired, as needed, or at a specific 
time interval, and are updated as information intended 
for the internal memory. 

[Twelfth Embodiment] 

An explanation will now be given for the processing 
performed in a case where, as in the eleventh embodi- 
ment, a color printer is busy when <file A> is to be print- 
ed. 

First, when "Print <file A>" is input, it is entered in 
the job table, and its object is understood to be a request 
that printing be performed. Assume as the condition/sit- 
uation that the colored portion is included in a document 
and a color printer is busy. A plan is devised to query a 
user concerning whether the job can wait until the color 
printer is not busy, or whether the data can be printed 
in monochrome. The query, "The color printer is busy. 
Wait, or print in monochrome?" is presented to a user. 
When the user selects "Wait" the process is placed on 
standby until the color printer is no longer busy. 

[Thirteenth Embodiment] 

When "<file A> was changed to <file AS" is input, it 
is ascertained that the updating of the job table is the 
object. As the condition/situation, the <file A> printing 
job is stored in the job table. Thus, a plan is made to 
query a user concerning the changing of the printing tar- 
get 1o <file A'>. Then, the query "Print <file A*> instead 
of <file A> before amended?" is presented to the user. 

[Fourteenth Embodiment] 

According to this embodiment, in a system wherein 
a plurality of apparatuses are connected together (in this 
case, they are connected across a network), when a job 
for an individual apparatus is instructed, an analysis of 
the object of the job is performed by the appearance. 
When the apparatuses determine from their statuses 
and the status of the instructed apparatus, that the in- 
structed apparatus should not execute the job, the per- 
formance of the job is transferred to another apparatus. 

Figs. 41 A and 41 B are diagrams in each of which 
is shown the condition where a plurality of apparatus are 
connected to a network. A network in Fig. 41 A provides 
an environment wherein a scanner 411 and printers 412 
and 413 can determine the states of the other appara- 
tuses. A network in Fig. 41 B provides an environment 
wherein the scanner 411 and the printers 412 and 413 
can not determine the states of the other apparatuses. 

Fig. 42 is a diagram illustrating a functional arrange- 
ment for performing the processing in this embodiment. 
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A job reception unit 421 receives a job from a user or 
from another apparatus. A received job is registered in 
a job table 422. A job analysis unit 423 extracts the job 
from the job table 422 and analyzes it. Another appara- 
tus information acquisition unit 424 determines the al- ^ 
tributes and the current statuses of other connected ap- 
paratuses. A self state determination unit 427 deter- 
mines the attribute and the current state of the self ap- 
paratus. 

An optimum planning unit 425 prepares an optimal 
plan for the performance of a job. A job performance 
determination unit 428 determines whether or not a job 
is to be performed, and whether the locally owned ap- 
paratus or another apparatus should perform the job. 
When the job is to be performed by the locally owned 
apparatus, a job performance unit 426 performs it. 
When another apparatus is to perform the job, another 
apparatus job transfer unit 429 transfers the job to an- 
other apparatus. A transfer notification unit 430 notifies 
a user of the results obtained by the job performance or 
the transfer of the job to another apparatus. Fig. 43 is a 
main flowchart showing the processing performed for 
this embodiment. 

At step S430, in order to determine whether a job 
has been input, a check is performed to determine 
whether or not a job has been input by a user, a job has 
been input by a device that can be detected by a system, 
based on the results obtained by analyzation of the jobs, 
or a new job that has been generated by the locally 
owned apparatus while idling. When a job has been in- 
put, at step S431 ; an input job is added to the job table. 
At step S432, the job table is examined to determine 
whether there is a job. If there is a job, at step S433 the 
job is extracted. At step S434 a received instruction is 
analyzed, and from the information attained by the an- 
alyzation, the object of the job is obtained. At step S435 
a check is performed to determine whether or not a job 
for another apparatus is present. If there is such a job, 
at step S436 control of another apparatus is initiated. If 
there is no such job, at step S437 a control process the 
locally owned apparatus is initiated. 

Fig. 44 is a flowchart for the process for controlling 
another apparatus performed at step S436 in the main 
flowchart. 

At step S440 the status ol an apparatus that is des- 
ignated in the job is examined, and at step S441 . based 
on the status, a check is performed to determine wheth- 
er a problem affecting the performance of the job by the 
designated apparatus exists. If no such problem exists, 
at step S442 the job is transmitted to the designated ap- 
paratus. If a problem affecting the performance of the 
job by the designated apparatus exists, at step S443 a 
check is performed to determine whether or not there is 
an apparatus other than the designated apparatus that 
is suitable for the performance of the job. If there is such 
an apparatus, its status is examined. As a result, if there 
is another apparatus that can execute the job, at step 
S445 the job is transferred to that apparatus, and at step 



S446 the user is notified of that the job was transferred 
to that apparatus. When there is no apparatus other than 
the designated apparatus that can perform the job, or 
when another suitable apparatus can not be used at that 
time, at step S447 an optimal plan is devised that will 
not degrade the object of the job, and at step S448, the 
plan is proposed to the user. 

Fig. 45 is a flowchart for the process for controlling 
the locally owned apparatus performed at step S437 in 
the main flowchart. 

At step S450 the status of the locally owned appa- 
ratus is examined, and at step S451, based on the sta- 
tus, a check is performed to determine whether a prob- 
lem affecting the performance of the job by the locally 
owned apparatus exists. If no such problem exists, at 
step S452 the job is performed by the locally owned ap- 
paratus. If a problem affecting the performance of the 
job by the locally owned apparatus exits, at step S453 
a check is performed to determine whether or not there 
is an apparatus other than the locally owned apparatus 
that is suitable for the performance of the job. If there is 
such an apparatus, its status is examined. As a result, 
if there is another apparatus that can execute the job, 
at step 5455 the job is transferred to that apparatus, and 
at step S456 the user is notified that the job was trans- 
ferred to that apparatus. When there is no apparatus 
other than the locally owned apparatus that can perform 
the job, or when another suitable apparatus can not be 
used at that time, at step S457 an optimal plan is devised 
that will not degrade the object of the job, and at step 
S458, the plan is proposed to the user. 

Fig. 46 is a diagram showing the condition that ex- 
ists when, while, in Fig. 41 A, an instruction is issued to 
output to the printer 412 along route A information that 
has been read by the scanner 411 , the other apparatus 
information acquisition unit 424 of the scanner 411 dis- 
covers a malfunction at the printer 412, and this infor- 
mation is output to the printer 413 along route B. 

Fig. 47 is a diagram showing the condition that ex- 
ists when, while, in Fig. 41 B, the scanner 411 is instruct- 
ed to output the information that is read to the printer 
41 2 along route A and to confirm the status of the printer 
412, since the printer 412 is located in a remote area, 
the scanner 411 takes the status of the network into ac- 
count and unconditionally transmits the information to 
the printer 412, which then, because a malfunction has 
occurred there, transfers the received information (job) 
to the printer 413. 

The processing performed in Figs. 46 and 47 will be 
described while referring to the flowcharts shown in 
Figs. 43 to 45. 

When in Fig. 46 a job is to be output by the scanner 
411 and transferred to the printer 412, the scanner 411 
performs the following process. 

First, at step S430 the scanner 411 receives a job 
for the scanning of information and the transmission of 
it to the printer 412. At step S431 the job is entered in 
the job table, and at step S433, it is extracted therefrom. 
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At step S434 the scanner 411 scans the data and as- 
certains the job involves the transmission ol data to a 
designated printer. At step S435 it is determined that 
there is a job. which is to be executed by another appa- 
ratus, for the output of the scanned information by the 
printer 412. Based on this determination, at step S436 
program control moves to another apparatus control 
process in Fig. 44. 

At step S440 the current state of the printer 412 is 
acquired. Then, information that the printer 41 2 has mal- 
functioned is received, and at step S441 it is determined 
that printing by the printer 412 is disabled. In this case, 
besides a malfunction, the printer 41 2 can also be in an 
out of paper or out of toner disabled state. At step S443, 
it is assumed that the designated printer 412 can not 
execute the job, and another apparatus that can execute 
the job is searched 1or by communicating with the other 
apparatuses across the network. At step S444 it is found 
that the printer 413 can execute the job and the perform- 
ance of the job is assigned to it. 

At step S445 the scanned information is transmitted 
to the printer 413. And at step S446, since a process 
other than the one instructed is being performed, the us- 
er is notified of the job alteration. Since the process for 
the instructed job has been terminated, the system waits 
for the next job. 

Following this, a process in Fig. 47 where a job is 
to be output by the scanner 411 and transmitted to the 
printer 412 will now be explained 

First, at step S430 the scanner 411 receives a job 
for the scanning of information and the transmission of 
the information to the printer 412. At slep S431 the job 
is entered in the job table, and at step S433, it is extract- 
ed therefrom. At step S434 the scanner 411 scans the 
data and ascertains the job is for the transmission of 
data to a designated printer. At step S435 it is deter- 
mined that there is a job, which is to be executed by 
another apparatus, for the output of the scanned infor- 
mation by trie printer 412. Based on this determination, 
at step S436 program control moves to another appa- 
ratus control process in Fig. 44. 

At step S440. while the scanner 411 communicates 
with the printer 412 and attempts to acquire its status, 
it is found that the printer 412 is in a remote area and 
the scanner 411 decides to transmit the information, re- 
gardless of the status of the printer 412. At step S441 it 
is determined that there is no problem with the printer 
412 since it is in a remote area, and at step S442 the 
scanned information is transmitted to the printer 412, 
which is the designated apparatus, and the job is trans- 
ferred thereto. Since the instructed job has been termi- 
nated, the scanner 411 waits for the next job 

At step S430 the printer 41 2 receives the job for out- 
putting the received information. At step S431 the job is 
entered in the job table, and at step S433, it is extracted 
therefrom. At step S434 the printer 41 2 understandsthat 
the job is for the printer 41 2 to output the received infor- 
mation. At step S435 it is determined that there is no job 



for another apparatus, and at step S437 program control 
moves to an own apparatus control process in Fig. 45. 

At step S450 the current state of the printer 412 is 
examined, and at step S451 the printer 412 judges by 

s itself that the printer 412 has a malfunction and is in a 
printing disabled state. In this case, besides the mal- 
function, the printer 412 can be in such an output disa- 
bled state as out of paper or out of toner. At step S453, 
another apparatus that can execute the job is searched 

10 for by communication with the other apparatuses across 
the network. 

At step 5454 the printer 413 that can execute the 
job is found, and the execution of the job at the printer 
41 3 is determined. At step S455, the printer 412 trans- 
15 mits the received information to the printer 413. At step 
S456, since the process other than being instructed is 
performed, the user is notified of the alteration of the 
job. In this embodiment, such a notice is issued to the 
instruction source user. Since the output destination is 
20 far away from the instruction source, the notice may be 
issued to a recipient. Since the instructed job has been 
terminated, the system waits for the next job. As at the 
printer 41 3 there is no processing problem and the print- 
er 41 3 can perform the job for the output of the received 
25 information, the printer 413 performs the job. 

In the above embodiment, the individual units auto- 
matically transfer the job. tf there is a problem at each 
apparatus : a user may provide a transmission order for 
the transfer of the job by the apparatuses. In this em- 
30 bodiment, a user inputs as a job the employment of a 
designated printer to print a scanned document; how- 
ever, a second and a third printer may at the same time 
be set 1or the job, so that if the designated printer mal- 
functions they can replace it and perform the printing. 
35 Thus, at step S443 or S453, the second orthe third print- 
er can be selected as a suitable apparatus for the job 
and its status is examined. A second or a third printer 
may be set as a printer to receive a job and to replace 
a designated printer when it is in a printing disabled 
40 state. However, even when a second or the third printer 
is not set for the job, the printer that receives the job can 
transfer it in consonance with destinations that are set 
internally. 

In addition to a malfunction and an operation disa- 
45 bled state, the following conditions can cause the per- 
formance of a job to be canceled: the specifications may 
be inappropriate for the performance of the job, e.g., a 
case where a job for the printing of color dala is issued 
to a monochrome printer, or immediate processing of 
50 the job may not be possible because there are too many 
jobs in a queue. 

[Fifteenth Embodiment] 

55 (n this embodiment, in a process for determining the 

status of the locally owned apparatus at step S450 in 
Fig. 45, schedule information for a user, which is the ob- 
ject, is examined as a factor that is used to determine 
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whether a problem exists that will affect the performance 
of a job involving the output to a user of urgent informa- 
tion received by a printer. 

An explanation will be given for a process that is to 
be performed in the environment shown in Fig. 48, 
where a user, to whom urgent information is to be trans- 
mitted, is currently at an outside location at which a print- 
er 412 is available that is connected to a network. 

In the procedures in Fig. 43, at step S434 it is as- 
certained that the object of the job is the transmission 
to a user of urgent information that has been received. 
At step S435, since at this time there is no job for another 
apparatus, program control moves to step S437. At step 
S437 the process advances to the locally owned appa- 
ratus control process in Fig. 45. 

At step S450 : in order to determine the status of the 
locally owned apparatus, the schedule of a user who is 
designated as a transmission source is confirmed by the 
PC 41 4, and it is ascertained that the user is at an out- 
side location. Since the user is at an outside location, at 
step S451 it is determined that a problem related to mak- 
ing urgent contact exists at the locally owned apparatus. 
At step S453acheck is performed to determine whether 
or not a printer that can output information is present at 
the user's destination. At step S454 it is confirmed that 
there is a printer 412 that can output information, and rt 
is determined that the printer 412 will be used to output 
the information. At step S455 the information is trans- 
mitted to the printer 412. At step S456 the users at the 
transmission source and at the reception destination are 
notified that the job has been transferred. 

When, at step S443, there is no printer available to 
which to transfer the job but there is another apparatus 
that can be employed by the user, such as a PC, a fac- 
simile machine or a telephone, at step S444 it may be 
determined to perform the job by using one of the avail- 
able apparatuses. Since the information can not be 
transmitted unchanged, at step S445 the information to 
be output to the printer is converted into an electronic 
mail document or a facsimile document, or is changed 
to voice information for the output. 

At step S446, depending on the medium used for 
the output, a notice that the job was transferred to an- 
other medium need not be issued, and can be transmit- 
ted by using the same medium. For example, when a 
telephone is used to transmit the information, it is better 
to notify a user of the job transfer before or after the 
transmission of the information. 

Instead of the telephone, at step S443, the above 
PC, the facsimile machine or the telephone may be de- 
termined to bea suitable apparatus to replacethe printer 
for the job. At step S447 the employment of such an ap- 
paratus may be determined to be an optimal plan that 
does not degrade the object of the job, and at step S448 
the plan may be proposed to a user. 



[Sixteenth Embodiment] 

Fig. 49 is a flowchart showing the processing for this 
embodiment. Steps 5498 to S500 are additionally pro- 
s vided for the flowchart in Fig. 43 as a process to be per- 
formed when there is no job. At step S498 the current 
operating statuses of other apparatuses are examined, 
and at step S499 a check is performed to determine 
whether or not a job that the locally owned apparatus 
can perform is being held in another apparatus and not 
being performed. If such a job is being held in another 
apparatus : at step S500 the job is extracted from the 
pertinent apparatus and is entered in the job table for 
the locally owned apparatus, while it is deleted from the 
job table for the pertinent apparatus. In this embodi- 
ment, therefore, an apparatus finds a job by itselt and 
processes it. 

Fig. 50 is a conceptual diagram showing the situa- 
tion for this processing. Since no job is present in a print- 
er 41 2, and a printer 41 3 has ten jobs in a queue, a print- 
er 41 5 acquires a job from the printer 41 3 and processes 
it. 

The process performed by the printer 415 is as fol- 
lows. At step S492 there is no job, and program control 
moves to step S498. At step S498 the current statuses 
of the other printers, 41 2 and 413, are examined. At step 
S499 it is determined that at the printer 413 there is a 
job that the printer 415 can execute. At step S500 the 
printer 415 extracts that job from the printer 413, and 
enters it in its job table. Then, the extracted job isdelefed 
from the job table for the printer 413. Since at step S490 
there is no job input, and at step S492 there is a job that 
was entered previously, program control moves to step 
S493 to execute the job. 

The operations for the performance of the job by the 
locally owned apparatus, which were explained in the 
fourteenth and the fifteenth embodiments, are per- 
formed, and the processing is thereafter terminated. 

[Seventeenth Embodiment] 

Fig. 51 is a diagram showing the transmission in this 
embodiment of a print job and other information in a sys- 
tem employing apparatuses. The heavy arrows are 
used to depict the transmission routes for the print jobs, 
and the broken line arrows are sued to depict the trans- 
mission routes for other information. 

The print job in this embodiment includes parame- 
ters that are referred to during printing: the document to 
be printed; the number of print copies; the print quality; 
and the print size. The other information includes the 
statuses, such as printing, normal, or out ol paper, of 
individual printers, and a schedule for a print job proc- 
essed by a print job scheduling unit. 

In Fig. 51, in a client machine 510, a print job gen- 
eration unit 512 generates a print job, and stores it in a 
print job memory unit 513. Then, the print job is trans- 
mitted by a print job transmission unit 514 in the client 
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machine 510 to a server machine 511 for managing 
printers. 

in the server machine 51 1 , a print job reception unit 
516 receives the print job. When the parameters of the 
print job have not yet been set, the print job is transmit- 
ted to a print job automatic setup unit 51 7, and when the 
parameters have already been set, the print job is trans- 
mitted to a print job automatic change unit 518. 

The print job automatic setup unit 517 sets print pa- 
rameters while taking into consideration the urgency, 
economy and the quality that are acquired by the print 
job. The print job automatic change unit 51 8 can perform 
the same process for changing parameters that have 
previously been set. The print jobs that are set or altered 
are registered in a print job schedule included in a print 
job scheduling unit 520 and are sequentially read to a 
print job transmission unit 519, or the print job may be 
directly transmitted to the printer job transmission on the 
unit 519 from the print job automatic setup unit 517 or 
the print job automatic change unit 518. The printing is 
then performed. 

The status, such as in printing, normal or out of pa- 
per, of an available printer is acquired by a printer state 
acknowledgement unrt 515, and is transmitted to the 
print job automatic setup unit 517 and the print job au- 
tomatic change unit 51 8 where it is employed for the set- 
up or change of the print job. Similarly, the printing 
schedule for the print job included in the print job sched- 
uling unit 520 is also transmitted to these units 51 7 and 
518 for their employment. 

As is shown in Fig. 51 , a print job may be transmitted 
along a different route. When a machine for generating 
a print job is the same as a machine for managing print- 
ers, the print job may not be transmitted via the print job 
transmission unit 514. 

Fig. 52 is a flowchart showing the processing per- 
formed by the print job automatic setup unit 517. The 
print job automatic setup unit 517 automatically sets pa- 
rameters within a range that satisfies a designated qual- 
ity and the printing can be performed as rapidly and as 
economically as possible. 

In Fig. 52, at step S520 a factor for determining the 
printing time and the amount of consumed toner is ini- 
tialized in preparing lor the simulation. Then, at step 
S521 a quality determination variable is initialized to a 
maximum quality value (e.g., 5), and at step S522 the 
period of time for the priming is simulated. At step S523 
a check is performed to determine whether the simulat- 
ed period of time for the printing equals a specified val- 
ue. When the time period for the printing equals a spec- 
ified value, at step S524the consumption of toner is sim- 
ulated. At step S525 a check is performed to determine 
whether the simulated consumption of toner equals a 
specified value. When the consumption of toner equals 
a specified value, the job is changed to the current qual- 
ity determination variable value at step S526. 

When, at step S523 or S525, the period of time for 
the printing or the consumption of toner is not equivalent 



to its specified value, at step S527 the quality determi- 
nation variable is decremented by one. At step S528 a 
check is performed to determine whether or not the re- 
sultant quality determination variable satisfies is equiv- 
s alent to a designated quality. When the designated qual- 
ity is satisfied, program control returns to step S522. 
When the designated quality is not satisfied, the 
processing is terminated with an automatic setup disa- 
bled state. 

Fig. 53 is a table showing the period of time for the 
printing and the consumption of toner for one character 
corresponding to different specified qualities for the 
printers A521 and B522. These values may be fixed val- 
ues inherent to individual printers, or may be dynamic 
variable values, which are averages obtained for print- 
ing periods or tor consumption of toner that were meas- 
ured in the past. 

Fig. 54 is a diagram showing the contents of a print 
job that is to be output in this embodiment, and param- 
eters that were designated by a user. In the system for 
the embodiment, the contents of a print job in Fig. 54 
and the set-up parameters are collectively called a print 
job. In this embodiment, calculated in terms of charac- 
ters, 500 characters are employed as the printing quan- 
tity for the print job in order to simulate the period of time 
for the printing and the consumption of toner. In addition, 
it is assumed that for urgency a user will designate a 
period of only one minute, and will keep the default val- 
ues for economy and quality. 

In the example shown in Fig. 54, when the print job 
automatic setup unit 517 is activated, at step S520 in 
preparing of simulation, a factor for determining the pe- 
riod of time for the printing and the consumption of toner 
is initialized. As defined in the table in Fig. 53, for exam- 
ple, the periods of time required for the printing of one 
minimum quality (= 1 ) character by the printer A521 and 
the printer B522 are, on average, 100 mS and 200 mS 
respectively, and the consumption of toner for one char- 
acter are 100 mg and. 200 mg. 

Following this, at step S521 the quality determina- 
tion variable is initialized to the maximum quality value 
(e.g., 5), and at step S522 the printing period of time for 
the printing is simulated. If, for example, the printers 
A521 and B522 are prepared for printing and no print 
job is scheduled in the print job scheduling unit 520, 
printing can be initiated immediately, and no waiting time 
is required. Thus, when the target print job includes 500 
characters, as simulation, printer A takes 250 seconds 
to print the characters and printer B takes 500 seconds. 

Then, at step S523 it is determined that this condi- 
tion does not satisfy the one minute urgency period that 
is specified by the parameter in Fig. 54, and at step S527 
the quality determination variable value is decremented 
by one. 

Since the quality is not specified in the parameter 
in Fig. 54, program control returns to step S522 for sim- 
ulation of the period of time for the printing When, as 
the result of the repetition of the above process, the 
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quality is set to the minimum quality ot 1 , the period of 
time for the printing by the printer A521 is 50 seconds, 
which satisfies the urgent figure. Therefore, the quality 
is set to the minimum quality determination variable val- 
ue (- 1). and the processing is normally terminated. 

If the quality is set to 2 or higher, it does not meet 
the parameters set by the user, and the processing is 
terminated as an automatic setup disable error. 

[Eighteenth Embodiment] 

Fig. 55 is a flowchart showing the processing per- 
formed by a print job automatic change unit 518. The 
print job automatic change unit 518 automatically 
changes a parameter within a range that satisfies a 
specified urgency and economy so as to print data hav- 
ing as high a quality as possible. When all the specified 
parameters can not be satisfied, the most important des- 
ignated parameter is satisfied and the others are 
changed as slightly as possible. 

Fig. 56 is a diagram showing the contents of a print 
job that is to be output in this embodiment, and param- 
eters designated by a user. In the system of the embod- 
iment, the contents of a print job in Fig. 56 and the set- 
up parameters are collectively called a print job. In this 
embodiment, calculated in terms of characters, 500 
characters are employed as the printing quantity for the 
print job in order to simulate the period of time for the 
printing and the consumption of toner. In addition, H is 
assumed that a user designates all of the parameters 
and particularly specifies that the quality is the most im- 
portant. 

In the example shown in Fig. 55, when the print job 
automatic change unit 518 is activated, at step S520 in 
preparing for the simulation , a factor for determining the 
period of time for the printing and the consumption of 
toner is initialized. As is defined in the table in Fig. 53, 
for example, the period of time required for the printing 
of one minimum quality (= 1) character by the printer 
A521 and the printer B522 are, on average, 100 mS and 
200 mS respectively, and the consumption of toner for 
one character are 100 mg and 200 mg. 

Following this, at step S551 the quality determina- 
tion variable is initialized to the maximum quality value 
(e.g. , 5), and at step S552 the period of time for the print- 
ing is simulated. If, for example, the printers A521 and 
B522 are prepared for printing and no print job is sched- 
uled in the print job scheduling unit 520, printing can be 
initialed immediately, and no waiting time is required. 
Thus, when the target print job includes 500 characters, 
as simulation, printer A takes 250 seconds to print the 
characters and printer B takes 500 seconds. 

Then, at step S553 it is determined that this condi- 
tion does not satisfy the one minute urgency period that 
is specified by the parameter in Fig. 56, and at step S556 
the quality determination variable value is decremented 
by one. 

Since, at step S557, the quality in the parameter in 



Fig. 56 is not satisfied, program control returns to step 
S552 for simulation of the period of time for the printing. 
As a result of the repetition of the above process, even 
when the quality is the one (= 3) designated by the user, 
5 the urgency period can not be satisfied. When, at step 
S556, the quality determination variable value is decre- 
mented by one, at step S557 the quality specified by the 
user is still not satisfied, and program control therefore 
moves to step S558. At step S558 a check is performed 
to to determine whether the quality is the most important 
parameter. When the quality is the most important pa- 
rameter, at step S560 the print job is changed in accord- 
ance with the specified quality, and the processing is 
normally terminated. 
75 For a case where the urgency is specified as the 
most important, when the quality is set to the minimum 
quality of 1, the printing period for the printer A521 is 50 
seconds, which conforms to the urgency setting. There- 
fore, the quality is set to the quality determination vari- 
20 able value (= 1), and the processing is normally termi- 
nated. 

[Nineteenth Embodiment] 

25 Fig. 57 is a diagram showing the transmission of a 

print job and other information in a system in this em- 
bodiment employing apparatuses. The thick arrows de- 
scribe the transmission routes for print jobs and the bro- 
ken line arrows describe the transmission routes for oth- 

30 er information. 

The print job in this embodiment includes parame- 
ters that are referred to during printing: a document to 
be printed; the number of print copies: the print quality; 
and the print size. The other information includes sta- 

35 fuses, such as printing, normal, or out of paper of indi- 
vidual printers, and a schedule for a print job that is in- 
cluded in a print job scheduling unit 520. 

In Fig. 57, in a client machine 510, a print job gen- 
eration unit 512 generates a print job and stores it in a 

40 print job memory unit 513. Then, the print job is trans- 
mitted by a print job transmission unit 514 in the client 
machine 510 to a server machine 511 for managing 
printers. 

In the server machine 511 , a print job reception unit 
is 51 6 receives the print job and transmits it to a print job 
interpretation unit 571. The print job interpretation unit 
571 interprets an instruction from a user that is included 
in the received print job, and converts the instruction into 
information, such as a parameter, that can be processed 
so by the system. When the parameters of the print job 
have not yet been set, the print job is transmitted to a 
print job automatic setup unit 517, and when the param- 
eters have already been set, the print job is transmitted 
to a print job automatic change unit 518. 
55 The print job automatic setup unit 51 7 sets print pa- 
rameters while taking into consideration the urgency, 
economy and quality values that are acquired from the 
, print job. The print job automatic change unit 518 can 
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perform the same process to change the parameters 
that have been set. The print jobs that are set or altered 
are registered in a print job schedule included in a print 
job scheduling unit 520 and are sequentially read to a 
print job transmission unit 519, or the print job is directly 
transmitted to the print job transmission unit 519 from 
the print job automatic setup unit 517 or the print job 
automatic change unit 518. The printing is then per- 
formed. 

The status, such as in printing, normal or out of pa- 
per, of an available printer is acquired by a printer state 
acknowledgement unit 515, and is transmitted to the 
print job automatic setup unit 517 and the print job au- 
tomatic change unit 518 where it is employed tor the set- 
up or change of the print job. Similarly, the printing 
schedule tor the print job included in the print job sched- 
uling unit 520 is also transmitted to these units 51 7 and 
518 for their employment. 

As is shown in Fig. 57, a print job may be transmitted 
along a different route. When a machine for generating 
a print job is the same as a machine for managing print- 
ers, the print job may not be transmitted via the print job 
transmission unit 514. 

In this embodiment, as is shown in Fig. 58A, the pa- 
rameters of the print job are set ambiguously by using 
natural language. Fig. 58B is a diagram showing, rela- 
tive to an input character siring in the natural language, 
the meanings represented by character strings accord- 
ing to the parameters and the ilems required to complete 
the meanings. The print job interpretation unit 571 refers 
to the meaning corresponding to an input character 
string in the natural language in Fig. 58B and the item 
required to complete the meaning, and interprets the in- 
struction given in the natural language that is received 
as a print job setup parameter as follows. 

From the character string "Print" in the input natural 
language, "printing" is the action that is the object of the 
user according to the meaning of the character string. 
Further, it is assumed that the required item "object" in- 
dicates the contents of a print job that was input at the 
same time, and that "quality" and "number of sheets" 
should be designated at a portion in the input natural 
language that has not yet been interpreted. Thus, the 
character string "one copy for client and five copies for 
members' is interpreted as an instruction to print one 
"sheet" having a high "quality" and five "sheets" having 
a desired "quality". 

The processes hereinafter perlormed by the print 
job automatic setup unit 517 and the print job automatic 
change unit 518 are the same as those in the seven- 
teenth and the eighteenth embodiments. 

[Twentieth Embodiment] 

In this embodiment, as is shown in Fig. 59, the print 
job parameters are set based on the distribution desti- 
nation described in a document, which is the contents 
of the print job, and on the number of sheets. 



The action of the job is printing. 

Printing for a "client" is interpreted as high quality 
printing, and the number of sheets is one. 

"ABC project" is interpreted as a "member use" and 
s the printing quality is arbitrary, the number of sheets be- 
ing five. A user acquires in advance information that he 
or she is a member of the ABC project. 

The processes hereinafter performed by the print 
job automatic setup unit 517 and the print job automatic 
to change unit 518 are the same as those in the seven- 
teenth and the eighteenth embodiments. 

[Twenty-first Embodiment] 

is Fig. 60 is a diagram showing the transmission of a 
print job and other information in a system in this em- 
bodiment employing apparatuses. The heavy arrows 
describe the transmission routes for print jobs and the 
broken line arrows describe the transmission routes for 

20 another information. 

The print job in this embodiment includes parame- 
ters that are referred to during printing: the document to 
be prinled; the number of print copies; print quality; and 
print size. The other information includes statuses, such 

25 as printing, normal, or out of paper, of individual printers, 
and a schedule for a print job that is included in a print 
job scheduling unit 520. 

In Fig. 60, in a client machine 510, a print job gen- 
eration unit 512 generates a print job, and stores it in a 

30 print job memory unit 513. Then, the print job is trans- 
mitted by a print job transmission unit 514 in the client 
machine 510 to a server machine 511 for managing 
printers. 

In the server machine 511 , a print job reception unit 

35 516 receives the print job and transmits it to a print job 
interpretation unit 571. The print job interpretation unit 
571 interprets an instruction from a user that is included 
in the received print job, and converts the instruction into 
information, such as a parameter, that can be processed 

40 by the system. When the parameters of the print job 
have not yet been set, the print job is transmitted to a 
print job automatic setup unit 517, and when the param- 
eters have already been set, the print job is transmitted 
to a print job automatic change unit 51 8. A print job sim- 

A5 ulation unit 601 simulates printing at the setup that is 
specified by the print job to acquire the period of time 
for the printing and the consumption of toner. 

The print job automatic setup unit 517 sets print pa- 
rameters while taking into consideration the urgency, 

so economy and quality values that are acquired from the 
print job. The print job automatic change unit 518 can 
perform the same process to change the parameters 
that have been set. The print jobs that are set or altered 
are registered in a print job schedule included in a print 

55 job scheduling unit 520 and are sequentially read to a 
print job transmission unit 51 9, or the print job is directly 
transmitted to the print job transmission unit 519 from 
the print job automatic setup unit 517 or the print job 
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automatic change unit 518. The printing is then per- 
formed. 

The status, such as printing, normal or out of paper, 
of an available printer is acquired by a printer state ac- 
knowledgement unit 515, and is transmitted to the print 
job automatic setup unit 517 and the print job automatic 
change unit 518 where it is employed for the setup or 
change of the print job. Similarly, the printing schedule 
for the print job included in the print job scheduling unit 
520 is also transmitted to these units 517 and 518 for 
their employment 

As is shown in Fig. 60, a print job may be transmitted 
along a different route. When a machine for generating 
a print job is the same as a machine for managing print- 
ers, the print job may not be transmitted via the print job 
transmission unit 514. 

Fig. 61 is a flowchart showing the processing per- 
formed by the print job simulation unit 601 . First, at step 
S611, the print job simulation unit 601 initializes, in pre- 
paring for the simulation, a factor for determining the 
printing period and the consumption of toner. At step 
S612 the period of time for the printing is simulated at 
the setup that is also specified by the print job, and at 
step S613 the consumption of toner is simulated at the 
setup designated by the print job. In this manner, the 
period of time for the printing and the consumption of 
toner are acquired. 

A specific method is the same as that explained in 
the seventeenth and the eighteenth embodiments. 

[Twenty-second Embodiment] 

In this embodiment, a plurality of output trays are 
provided for a printer, and a tray is selected in conso- 
nance with a user. 

Fig. 62 is a flowchart showing the processing for a 
printer system in this embodiment. When a printer sys- 
tem receives a print job, first it performs a process for 
coping with a case where a plurality of print jobs are re- 
ceived at the same time, and then it initiates the process- 
ing shown in Fig. 62. 

Figs. 63A and 63B are diagrams showing the exter- 
nal appearances of the printer systems in this embodi- 
ment. As is shown, printers 631 and 632 each have a 
plurality of output trays: tor the printer 631 , each output 
tray is provided with a screen for displaying the name of 
the person to whom a document on the tray belongs; 
and for the printer 631 , the names of the persons whose 
document remain on which trays are displayed on a 
common screen. When a user is set in advance for each 
output tray, the individual trays are user dedicated trays. 
When the trays are not specifically assigned, the users 
of the output trays are changed as needed. 

The printer system in this embodiment implements 
the functions of notifying a user of the receipt of a doc- 
ument, the termination of printing, and the failure to col- 
lect a printed document (uses a sensor). 

The above processing will now be described. In Fig. 



62, at step S621 information concerning a destination is 
acquired from information included in a print job. The 
information concerning the destination is obtained, as is 
explained in the twenty-ninth embodiment, either by in- 

5 terpreting the setup described in the print job, or by ex- 
tracting it from information, such as E-mail, that is set 
separately from the contents of the print job. At this time, 
when, as in conventional use, a printer is employed as 
an output device for a personal computer, usually an ad- 

to dressee is not designated. When the printer is employed 
as a facsimile machine or for transmission of E-mail, as 
in the system in this embodiment, an addressee is nor- 
mally designated. 

At step S622 a check is performed to determine 

is whether or not an addressee is designated. At step 
S623 the addressee is notified of the arrival of a docu- 
ment. At step S624 the addresser information is ac- 
quired from the information included in the print job. The 
addresser information is acquired, as is explained inthe 

20 twenty-ninth embodiment, either by interpreting the des- 
ignation described in the print job, or by extracting it from 
information, such as E-mail, that is set separately from 
the print job. When, as in conventional use, a printer is 
employed as an output device for a common personal 

25 computer, normally the addresser falls within the control 
sphere of the system. However, when the printer is used 
as a facsimile machine, as in the system in this embod- 
iment, usually the addresser is not within the control 
sphere. At step S625 a check is performed to determine 

30 whether the addresser is in the control sphere. At step 
S626 when the addressee is designated, a correspond- 
ing output tray is selected. When the addressee is not 
specified and only the addresser is specified, an output 
tray corresponding to the addresser is selected At step 

35 S627 the addresser or the addressee information is 
transmitted to the corresponding output tray, as is 
shown in Fig. 63A. Or, as is shown in Fig. 63B, the ad- 
dresser or the addressee information and the output tray 
that is employed are displayed on the common display. 

40 When the user is set for each output tray, the individual 
trays are user dedicated trays. When the trays are not 
specifically assigned, the users of the output trays are 
changed as needed. 

At step S628 printing is executed. At step S629 a 
check is performed to determine whether the printing 
has been terminated and whether the addresser is with- 
in the control sphere. At step S630 the addresser is no- 
tified ol the termination of the printing. At step S631 a 
check is performed to determine whether a document 

so has been left for a predetermined period of time and 
whether the addresser is in the control sphere. At step 
S632 the addresser is notified that a printed document 
has been left in the tray for an extended time period. 

55 [Twenty-third Embodiment] 

In this embodiment, information concerning the 
, time is handled. Fig. 65 is a diagram showing the func- 
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tionai arrangement of a system according to the embod- 
iment. A processor 650 includes an understanding unit 
652, a planning unit 653, an execution unit 654, a re- 
sponse unit 655 and a knowledge base 656, and is con- 
nected to a database 651 . 5 

Fig. 64 is a flowchart showing the processing per- 
formed by the functional arrangement in Fig. 65 in this 
embodiment. 

At step S640 a check is performed to determine 
whether or not there is external input. If so, at step S641 i<> 
the understanding unit 652 employs the knowledge 
base 656 to analyze an input document as the units of 
words and sentences, and at step S643 the object of the 
input contents is analyzed and ascertained. At step 

5644 a check is performed to determine whether infor- is 
mation required for an understanding of the object is in- 
sufficient. . if the information is unsatisfactory, at step 

5645 a query is issued to a user, and a response from 
the user is studied in the knowledge base 656. Program 
control thereafter returns to step S643. If the information 20 
is adequate, program control moves to step S646. 
When, at step S640, there is no input, at step 5642 a 

job to be executed is found and program control advanc- 
es to step S646. 

At step S646 the planning unit 653 prepares a plan 2S 
to achieve the object. At step S647 a check is performed 
to determine whether information required for planning 
is insufficient. If the information is unsatisfactory, at step 
S64S a query is issued to the user, and a response from 
the user is studied in the knowledge base 656 Program 30 
control thereafter returns to step S646. !f the information 
is adequate, at step S649 the execution unit 654 exe- 
cutes the plan. At this time, the execution unit 654 ac- 
cesses the database 651 or communicates with another 
application, as needed. At step S650 the response unit 35 
655 determines whether or not a response should be 
made. If a response should be made, a decision is made 
concerning the selection of the contents for the re- 
sponse. Further, at step S651 a response is prepared 
in accordance with the selected contents, and is trans- *o 
mitted to the user. 

The understanding unit 652 acknowledges, as 
needed, externally input information, such as keyboard 
input, voice input, E-mail or facsimile or news informa- 
tion, analyzes the received information and also the time 45 
concept contained in the words in sentences, and ob- 
tains an understanding of the contents of the sentences 
that are concerned with time. 

Fig. 67 is a diagram showing an example where 
schedule information is extracted from electronic mail, so 
In the above process, specifically, as is shown in Fig. 
67, the contents of a document received by E-mail are 
analyzed, and the time concepts such as "February 
22nd", "13:00" and "15:00" are found and analyzed. 
Then, an understanding of the sentence concerning the ss 
concept, "We have scheduled a patent system explana- 
tion meeting at conference room B from 13:00 to 15:00 
on February 22" is obtained, and an action associated 



with the time can be correlated with the schedule. 

The analyzed time concept is compared with the 
current time. If the time concept describes the future, a 
corresponding action is registered in the schedule, but 
if the time concept describes the past, it is abandoned 
without being registered or it is registered as an infor- 
mation of a type other than that for a schedule, such as 
information for a personal data base, that is used to 
record past events. 

The understanding unit 652 analyses the contents 
of externally input information, such as keyboard input, 
voice input, E-mail, facsimile or news information. When 
the understanding unit 652 finds in the sentences a term 
(a word or a concept) that is unknown to the system, or 
a plan that the system has not yet executed, and when 
the word or the plan that is detected is not urgent, the 
understanding unit 652 studies the term or the plan by 
sequentially searching for information concerning it, and 
stores it as knowledge in the knowledge base 656. 

Specifically, when, for example, the unknown term 
XYZ appears, the presence of XYZ is memorized; and 
sentences "XYZ is very delicious" and "XYZ is hard" are 
analyzed to extract the concept that XYZ is hard, deli- 
cious food. 

Similarly, when a new plan is instructed and when 
from the current situation it is understood to be a plan 
for use when a user is absent, the understanding unit 
652 studies the plan as an example of how to make a 
plan for when a user is absent. 

In addition, when, for example, a system has been 
acquiring information concerning a user from news and 
notifying the user each time such information is found, 
and then upon the receipt of a notification that the user 
responds by sending the system an instruction in natural 
language that "from now on, there is no need to take 
notice of this information 0 the system can ascertain that 
transmission of the information is no longer necessary. 

As is described above, in the system, the under- 
standing unit 652 analyzes the contents of externally in- 
put information and also analyzes the object. The plan- 
ning unit 653 prepares a plan of an action for the system 
to take to achieve the object. The execution unit 654 per- 
forms the processing. And the response unit 655 notifies 
a user of the results of the processing. 

When, for example, it ts ascertained, from informa- 
tion concerning an addresser, upon the receipt of the E- 
mail message shown in Fig. 67 that the mail is from an 
addresser who is not registered in the database, infor- 
mation concerning the addresser is extracted from the 
message and is registered in the personal database in 
the database 651 . Further when more information is ob- 
tained concerning the addresser who is not registered 
in the personal database, the information may be regis- 
tered in the personal database. 

Since the understanding unit 652 understands the 
message is for a notice for an explanation meeting, it 
accesses the schedule ol a user in the database 651 , 
and compares new schedule data with the current 
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schedule data. When the new schedule data and the 
current schedule data do not conflict, the new data are 
additionally re.: stered in the schedule. However, as is 
shown in Fig. when the schedule data conflict, a re- 
ply to the addresser and a notice for a user are prepared. 

As is described above, since this system under- 
stands the contents of received information, the system 
can prepare a response in consonance with the situation 
and execute it. 

In the above explanation, the system prepares a re- 
ply to a user who sent the E-mail without confirming it 
with a user and transmits the reply. This is because the 
sentence, "Please contact me as soon as possible if 
there is any difficulty" is in the message, and as a result 
of the analysis of this sentence, it is determined that a 
response is very urgently required. If the urgency of a 
response is determined to be low because a date for the 
holding the meeting is fairly advanced and the attend- 
ance at the meeting is arbitrary, the system confirms it 
with a user before transmitting a response. In other 
words, the system determines in which action to take in 
consonance with the degree of urgency. 

In addition, the schedule or the personal database 
can be accessed to transmit a response, relative to the 
input of the natural language by a user, as well as a nat- 
ural language instruction received from a user. 

For example, the personal database can be ac- 
cessed by asking the natural language question, "What 
is the telephone number of Mr. XX?". Or the schedule 
can be accessed when the natural language question, 
"Where will the meeting on the 16th be held?" is re- 
ceived, to send a reply to a user. 

Fig. 66 is a diagram showing the input/output types 
between the system that performs the overall process- 
ing in this embodiment and external devices. 

Input data can be data input via a keyboard, natural 
language information received by E-mail, documents or 
pictures input as images by a scanner, voice input via a 
microphone or images input by a camera. A character 
recognition process is preformed for a document that is 
read by the scanner, or a voice recognition process is 
performed for speech, so that input information in natu- 
ral language can be obtained. 

As for when there is no input (idle state), as is shown 
at step S642 in Fig. 64, when there is no external input 
the system searches for a job that it should execute and 
regards it as input. In the idle state, when no job is given 
to the system, the system, for example, accesses news 
and aggressively acquires as input information concern- 
ing the user. 

As for output, there are the registration of a sched- 
ule in a database, the filing or erasure of data relative 
to a file memory device, or the dispatch ol a notice to a 
user or a reply to an addresser. The importance of the 
output is that it can be determined that a process can 
not be performed within the control range of a system, 
and another action, such as transmitting a response to 
that effect, can be performed. 



An input/output destination is a user or an external 
device, or a different processor in the system or another 
application. 

In addition, input documents are analyzed, and data 
5 concerning a person and data concerning the time are 
extracted and stored in the personal database and the 
schedule database. Also, data required for the anticipa- 
tion of an action is extracted to use for planning. 

in the example in Fig. 67, as a result of analysis of 
1- an E-maii document that is input, the following descrip- 
tion concerning a person is acquired: 

To: toshima@abc.canon.co.jp, 
rohra@abc, canon.co.jp, 

is kazuyo@abc.canon.co.jp 
From ichiro@abc.canon.co.jp 
I am Suzuki of system promotion section. 
Suzuki (ichiro@abc.canon.co.jp) 
044-123-4569 (ext. 654-3210) 

20 Canon Inc. Intellectual Property Head Office Sys- 
tem Promotion Seci. n 

Assuming that a pei oon whose mail address is "ka- 
zuyo@abc.canon.co.jp" is already registered in the 
personal database and the other persons are not 
25 registered, as is shown in Fig. 67, data for PERSON 
1 , 2 and 3 are extracted and registered in the data- 
base 651. 

Since there is a description concerning the time, 
30 "We have scheduled a patent system explanation meet- 
ing at conference room B from 13:00 to 15:00 on Feb- 
ruary 22" data for EVENT1 is extracted. 

Further, since there is a description used for esti- 
mating an expected action, "Please contact me as soon 
35 as possible if there is a difficulty" this is used for the 
processing to be explained while referring to Fig. 70. 

Fig. 68 is a specific flowchart for step S649 pre- 
pared by extracting the portions from the flowchart in 
Fig. 64 that are required for performing the processes 
40 in Figs. 67 and 69. 

The processing will now be explained while refer- 
ring to Figs. 67 and 69. 

At step S680 E-mail is input and at step S681, of 
each word and sentence, the input E-mail document is 
45 analyzed. At step S683 it is ascertained from a signature 
and a header that "there is information concerning per- 
sons" and "an event, an explanation meeting : will be 
held". Further, it is ascertained that "a reply is requested 
if there is a difficulty*. 
so At step S684 the data concerning a person is ex- 
tracted and a plan for registering the data in a database 
and a plan for registering the explanation meeting event 
in the schedule are made. At step S685, it is ascertained 
that there is a plan to be executed. At step S686 it is 
55 found that there is no problem with the plan, and pro- 
gram control thereafter moves to step S688. At step 
S688'the data concerning the person is extracted and 
registered in the database, and program control returns 
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to step S685. 

In Fig. 67 ; at step S685 there is a plan to be exe- 
cuted: registration of the explanation meeting event in 
the schedule. At step S686 it is assumed that the event 
can be registered in the schedule with no problem, and 
program control goes to step S68S. At step S688 the 
event is registered in the schedule, and program control 
returns to step S685. At step S685 there is no other plan 
to be executed, and at step S689 no acknowledgement 
receipt is required. The processing is thereafter termi- 
nated. 

In Fig. 69. at step S685 there is a plan to be exe- 
cuted: registration of the explanation meeting event in 
the schedule, and at step S686 a conflict is found in the 
schedule. Since there is a problem with the execution 
of a plan, at step S687 a reply to that etlect is transmitted 
and a plan is made to notify a user that there is a prob- 
lem. At step S685 there is a plan for writing a reply and 
transmitting it, and at step S686 no problem concerning 
the execution of the plan is found. At step S688, there- 
fore, a reply indicating that the timing is inconvenient is 
prepared and transmitted to the addresser. At step S685 
there is a plan for preparing a document for notification 
of a user. At step S686, there is no problem in execution, 
and at step S688 a response to a user is prepared de- 
scribing that a reply was sent for reconsideration be- 
cause of the contents of the E-mail and the conflict of 
the schedules. At step S685 there is no plan to be exe- 
cuted and at step S689 since there is an notice to a user, 
it is transmitted to the user. The processing is thereafter 
terminated. 

In the example in Fig. 69, when the schedule infor- 
mation EVENT1 that is extracted is compared with the 
schedule EVENT2 that is planned previously, it is found 
that the two events conflict. Further, according to the in- 
formation extracted in the example in Fig. 69, it is ap- 
parent that a reply is necessary if there is a difficulty. In 
the system in this embodiment, the knowledge for pre- 
paring letter is employed to write a letter describing that 
a user can not attend the meeting because the sched- 
ules are conflicting, and the reply is automatically trans- 
mitted. 

In addition, the user of the system in this embodi- 
ment is notified that the system automatically has trans- 
mitted a reply for the user. 

[Twenty-fourth Embodiment] 

Fig. 70 is a diagram showing an example where a 
user sends a question by voice to a system according 
to this embodiment. 

When a user asks by voice "where will tomorrow's 
conference be held?", the system can examine the 
schedule information of the user and tell the location of 
the conference. 

When the user does not know how to get the loca- 
tion for the conference and asks D ln which area in Shi- 
momaruko?", the system examines the database, per- 



forms planning for an explanation to make a user under- 
stand, prepares a briefing story, and uses graphical im- 
ages and sounds to explain how to get the location. 
Fig. 71 is a flowchart showing the processing per- 
5 formed in Fig. 70 by referring to the basic flowchart in 
Fig. 64. An externa! input is regarded as a query. 

The processing relative to a first inquiry will be ex- 
plained. At step S710 voice (where will tomorrow's con- 
ference be held?) is externally input, and at step S711 
10 the input sentence is analyzed. At step S7 12 it is under- 
stood that the input sentence is an inquiry for the con- 
ference, and that the object of the user is to know the 
location of the conference. At step S713 the following 
planning is performed to answer the inquiry. (1 ) Tomor- 
is row's schedule is extracted. (2) The location registered 
in the schedule is acquired. (3) A reply is prepared to 
tell the acquired location. 

At step S714 information is sufficient and program 
control moves to step S716. If the schedule show a plu- 
20 rality of conferences, the system does not identify which 
conference is. Therefore, at step S715 a query is sent 
to a user to remake a plan. At step S716 the plan made 
at step S713 is executed, and a reply document to the 
user is prepared. At step S71 7 it is ascertained that there 
2S is a response to the user, the system determines the 
transmission of the response. At this time, since it is un- 
derstood that the input was done by voice, transmission 
of the reply by voice is also determined. At step S718 
the response, which is voice data obtained by conver- 
se sion, is transmitted to the user. 

The processing is temporarily terminated, and fol- 
lowing this, an inquiry from the user is input. The process 
relative to the second inquiry will now be described. 
At step S710, voice (in which area in Shimomaru- 
35 ko?) is input externally, and at step S711 the input sen- 
tence is analyzed. At step S71 2 it is understood that this 
inquiry is a continuous question of the first one, and also 
that a specific location of the place name that was an- 
swered is being asked. 
40 At step S713 the following plans are made to re- 
spond the question: (1) a plan for explaining that it is 
difficult to explain the location by writing; (2) a plan for 
explaining the location by sequentially sending images; 
(3) a plan for acquiring necessary images from an image 
45 database and a location database; and (4) a plan for 
preparing sentences corresponding to images. 

At step S714 the information is adequate, and at 
step S71 6 the plans made at step S71 3 are sequentially 
executed and a reply to a user consonant with the *m- 
so ages is prepared. At step S717, since there is a reply 
document to be transmitted to the user, the transmission 
of the response is determined. At this time, it is under- 
stood that the input was done by voice, so that it is also 
determined that explanation will be given by voice while 
ss displaying images on a monitor. At step S718 the re- 
sponse is transmitted to the user. The processing se- 
quence in Fig. 70 is thereafter terminated. 
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[Twenty-fifth Embodiment] 

In an example shown in Fig. 72, when two sched- 
uled events conflict, the order of priorities of the two 
events is obtained from a comparison, and a plan to han- 
dle this problem is prepared and proposed to a user 

That is, in this example, when one schedule is found 
to be more important than the other, it is proposed that 
the less important schedule be canceled. 

If the user knows somebody to send as his or her 
proxy to the scheduled event for which cancellation is 
proposed, the user can send the system a response to 
that effect. The system can then perform an action in 
consonance with the response. Also, the system can 
study a plan for despatching another person as a proxy. 

In Fig. 72 is shown a combination of other methods 
performed at step S646 in Fig. 64 to make plans when 
the schedule conflict is found at step S686 in the flow- 
chart in Fig. 68. 

The processing in Fig. 68 has been explained for 
unconditionally transmitting to a user a response for an 
inconvenience. In this embodiment, the processing for 
a plurality of plans is employed as the re-planning meth- 
od at step S646. 

Fig. 73 is a flowchart for evaluating a priority extract- 
ed from the contents of a document and for proposing 
it to a user. The processing will now be explained. 

The priorities of the scheduled events is determined 
by using information that the user has set in advance for 
each schedule, attendance at an event included in the 
schedule, the object of a scheduled event and the result 
obtained as a result of the analysis of the received doc- 
ument. A user may set the priorities of persons that are 
expected to attend the pertinent event. 

The priority determined according to the object of 
the schedule may be set in advance by a user, or may 
be determined while taking into account the object or 
the field that the system understands, from the study of 
the past, that the user is interested in. That is, when the 
user engages in computer associated work, the sched- 
ule for a computer associated event priority over an 
event associated with real estate. Of course, if a user 
plans to purchase a house and that data is stored as 
information concerning the user, the schedule of an 
event associated with real estate may take priority. 

For the priority determined from the result of the 
analysis of the document, when a document including 
the sentence "Please be sure to attend the meeting" and 
a document including the sentence "Please attend the 
meeting if possible" setup a conflict, the first document 
received is determined to take priority. 

In Fig. 73, at step S730 the current schedule infor- 
mation is acquired. At step S731 the priority for new 
schedule information is evaluated. At step S732the pri- 
orities of both schedules are compared. At step S733 
from the result of the comparison, a plan is made to pre- 
pare a notice for querying a user (e g , display two con- 
flicting scheduled events and ask whether the event 



having the lower priority can be canceled). At step S734 
the prepared notice is transmitted to the user. At step 
S735 a response from the user relative to the inquiry 
concerning the plan is acquired The same re-planning 
5 process as in Fig. 68 is performed to again prepare a 
. plan by using the response obtained from the user so 
that an action consonant with the response of the user 
can be taken. 

Fig. 74 is a flowchart showing the re-planning proc- 

io ess. 

At step S740 the response finally obtained in Fig. 
73 is regarded as input. At step S741 the input sentence 
is analyzed. At step S742 it is ascertained that the 
schedule is to be changed and that this is the first in- 
is structed plan. 

At step S743 the following plans are made: (a) the 
changing of the schedule; (b) the preparation of a doc- 
ument to transmit the schedule 1o a proxy; (c) the study 
of a new plan; (d) the preparation of a document to trans- 
it* mil the new plan to the user; and (e) the notification sent 
to a user concerning another schedule that should be 
recalled. 

At step S744 the plans made at step S743 are ex- 
ecuted: (a) the schedule is changed; (b) a document is 
25 prepared to transmit the schedule to a proxy; (c) a new 
plan is studied; (d) a document is prepared to transmit 
the new plan to the user; and (e) a user is notified of 
another schedule that should be recalled. 

At step S745 the notification document is transmit- 
30 ted to the user The processing for performing the proc- 
ess in Fig 72 is thereafter terminated. 

[Twenty-sixth Embodiment] 

35 Fig. 75 is a diagram illustrating the overall image of 

a system according to this embodiment, including input/ 
output devices. Fig. 76 is a flowchart showing the 
processing performed by the system in Fig. 75. 

Information input by E-mail, by voice, via a key- 

40 board, across the WWW, and by input devices, such as 
a telephone, a facsimile machine, a scanner and a cam- 
era, is analyzed by an input management unit 751 (step 
S760). A core unit 752 ascertains the contents of the 
information and plans an appropriate process (step 

45 S761). An output management unit 753 determines an 
output medium and prepares the contents to be output 
and an output route (step S763). Then, the information 
is output by E-mail, by voice, across the WWW, or by 
output devices, such as a telephone., a facsimile ma- 

50 chine, a printer and a copier. 

Figs. 77 to 79 are specific flowcharts showing the 
process at step S760 (the process performed by the in- 
put management unit 751), at step S761 (the process 
performed by the core unit 752) and at step S763 (the 

55 process performed by the output management unit 
753). 

In Fig. 77, when, at step S770, it is determined that 
, there is new input, at step S771 the input information is 
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obtained. At step S772 the input information is analyzed, 
as needed, after the information has been identified. At 
this time, the input management unit 751 accepts infor- 
mation from various media and identifies or analyzes the 
data using methods corresponding to the individual me- 
dia. 

In Fig. 78, the core unit 752 receives the information 
that is analyzed in Fig. 77. 

At step S780 the object of the input information is 
analyzed and ascertained from the result of the analysis. 
At step S781 how the object of the input information is 
related to the main apparatus (the system or the user) 
is examined. At stepS782, from the relationship with the 
main apparatus, planning for what to execute is per- 
formed in consonance with the object. At step S783 the 
plan made at step S782 is executed. 

In Fig. 79 the output management unit 753 employs 
the results obtained in Fig. 78 to determine whether a 
response is needed. If a response is required, it is pre- 
pared and output. 

At step S790 the result obtained in Fig. 78 is ana- 
lyzed. At step S791 a check is performed to determine 
whether or not there is a response relative to the results. 
When there is no response, the processing is thereafter 
terminated. If there is a response, at step S792 a re- 
sponse relative to the external response is determined. 
At step S793 a medium for the response is determined. 
At step S794 a response is prepared in consonance with 
the designated medium. At step S795the prepared re- 
sponse is issued by the designated medium. At step 
S896 the response is actually output. The processing 
performed by the arrangement in Fig. 75 has been ex- 
plained. 

When, for example, E-maii is received that notifies 
the user of the holding of a conference, the input man- 
agement unit 751 analyses the input information accord- 
ing to the procedures in Fig. 77. From the obtained re- 
sult, the core unit 752 prepares a plan, according to the 
procedures in Fig. 78, as to how the user and the system 
should handle the information. The plan is transmitted 
to the output management unit 753, which in turn per- 
forms the process according to the procedures in Fig. 
79. When a response should be transmitted with the 
plan, a response is actually prepared and transmitted. 

Depending on the contents of the conference de- 
scribed in E-mail message and the status of the sched- 
ule of the user, at step 5782 a plan is prepared for the 
aggressive transmission of the response to the user, so 
that preparation of the response is required. At step 
S791, therefore, it is determined that a response is to 
be transmitted. At step S792 the contents of the re- 
sponse are determined. When the schedules conflict 
and when it is uncertain which schedule should be se- 
lected or whether a user intends to attend a specific con- 
ference, the contents of a response asking for a decision 
are determined. At step S793 a medium is selected by 
which the contents of the response can be transmitted 
most effectively. When, for example, the user is outside 



the office, a medium, such as a telephone or a facsimile 
machine, is selected that can issue a notice to the user. 
When the user is being operating a personal computer, 
a medium, such as a personal computer, is selected that 

5 can effectively display various reference materials. At 
steps 5794 and S795 a document and an image are pre- 
pared in consonance with the contents and the medium 
that are decided above, and a response is prepared us- 
ing a method for converting the data into voiced natural 

io language in consonance with a specific medium, and is 
issued. 

Fig. 80 is a detailed flowchart for Fig. 75 to explain 
the processing for extracting data from input informa- 
tion. In this system according to the embodiment, when 

is a document is input, at step S801 type identification 
symbols, such as the layout and a bar code of a docu- 
ment, are employed to infer the document type. If the 
document is interred to be a letter, a report or a patent 
publication, program control moves to step S803. When 

20 no identification of the document is possible, program 
control goes to step S810 whereat OCR is thoroughly 
performed to ascertain the document type. 

At step SB03, in order to confirm the inferred docu- 
ment type is correct, a characteristic block is interpreted 

25 by the OCR to identify the document type (see Figs. 82A 
and 82B) At step S804 the addressee of a letter and 
the patent serial number of the patent publication are 
found and the document type is confirmed. When the 
confirmed document type matches the inferred docu- 

30 ment type, program control advances to step S806, If 
the document types do not match, program control 
moves to step S810. For slips, at the top of which char- 
acter strings describing the slip type are printed, the doc- 
ument type can be easily determined by the OCR on the 

35 employing upper portion of the sheet. 

At step S806 the knowledge base of the document 
type that has been confirmed is employed to read and 
interpret a specific block using OCR. At step S807 a 
check is performed to determine whether there was in- 

40 formation in the past that concerns the resultant infor- 
mation. If such information was present, program con- 
trol moves to step S808. If such information was not 
present, program control goes to step S810. As a result, 
the addressee or the patent number is found and wheth- 

45 er the information is a reply to a letter that was sent in 
the past can be ascertained. 

At step S808, based on the identified status, the ob- 
ject is decided from the contents of the important text, 
and at step S809 a process to be executed is deter- 

so mined. 

At step S811, because of the above object, the in- 
formation is analyzed in another range where OCR has 
not yet been performed. At step S812 a required proc- 
ess is actually performed. As a result, a filing process, 
55 etc., is performed as needed. 

In the above process, the OCR can be performed 
while inferring the contents of the information, more ef- 
ficient and precise OCR results can be obtained than 
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when, as in the conventional case, the OCR use is ef- 
fected from the beginning. Accordingly, the. process 
based on the OCR results can be performed precisely. 

Fig. 81 is a diagram showing an example letter/fac- 
simile that is a target for data extraction. 

In this example, at step S801 the document type is 
inferred from the layout of a document. As a result, Doc- 
Typel = letter/fax. 

At step S803 a specific block is read by the OCR 
and is interpreted (see Figs. 82A and 82B). Then, 
"From", "To" and "Dear Sir" are acquired, and at step 
S804 DocType2 = letter/fax. 

The process for reading a specific block by using 
OCR, which was explained at step S803 in Fig. 80, will 
be specifically described. Figs. 82A and 82B are dia- 
grams for explaining this process. 

The system in this embodiment scans a specific 
block as follows: 

(1 ) A specific block is read fast by pre-scanning with 
a low resolution. 

(2) A form of the information that is read is com- 
pared with an information form stored in a DB 823. 
When the forms match, input information other than 
the form, or information in an area (e.g., inside the 
frame of a card in Figs. 82A and 82B) specified for 
each form, is read at a high resolution. 

(3) Only the area that is read is regarded as an OCR 
target and analyzed, and the processing is contin- 
ued. 

In addition, since in this example it is assumed that 
there are a plurality of cards having the same form, doc- 
uments that are to be read at a predetermined interval 
probably have the same form. Therefore, first, the proc- 
ess is performed while it is assumed that the documents 
have the same form, and when a contradiction occurs, 
it is assumed that the documents have different forms 
and a re-analysis of them is performed. 

As a result, the processing speed is considerably 
increased, the analysis range is limited, and an analysis 
domain is determined by specifying a form, and the anal- 
ysis precision is drastically improved. 

The determination of the object from the contents 
of the information, which was explained at step S808 in 
Fig. 80, will be specifically described. 

The system in this embodiment employs date data 
to perform the determination process according to the 
procedures in Figs. 83A and 83B. When a sender is a 
user, the following process is performed. 

(1 ) When the current date is today, transmission by 
facsimile is decided. 

(2) When the date is yesterday or some time in the 
past, the user is queried as to whether the same 
document was received before or whether he or she 
has seen.the same contents, in order to determine 
whether to perform filing, re-transmission, or wheth- 



er the user made a mistake. 

(3) When the date is tomorrow or some day in the 
future, the user is queried in order to determine 
whether the information should be held until the 
5 designated date, or whether the user made a mis- 

take. 

More specifically, assuming that in a document in 
Fig. 81 : Datel is the date of a document, Date2 is the 
io today's date, 

(1) Datel = November 25, 1996 & Date2 = Novem- 
ber 25, 1996 - send the document 

(2) Datel = November 25, 1996 & Date2 = July 2, 
75 1996 --filed 

(3) Datel = November 25, 1996 & Date2 = Novem- 
ber 26, 1 996 

Date of contents = December 18 and 22, 1996 -- 
20 mistake 

Figs. 83A and 83B are flowcharts showing the 
processing for analyzing the object extracted from an 
input document. When a document is input, at step S830 
a check is performed to determine whether the sender 

2S js a user. If the sender is a person other than a user, the 
input document is determined to be a received docu- 
ment. At step S831 a corresponding object, such as fil- 
ing or data extraction, is examined, and the processing 
is thereafter terminated 

30 When the sender is a user, program control advanc- 

es to step S832 to analyze the type of the input docu- 
ment. At step S833 the date of the document is com- 
pared with the current date. When the two dates are 
close to each other, at step S835 a check is performed 

35 to determine whether the document was transmitted be- 
fore. When the document was not transmitted before, 
its object is determined to be "transmission of a docu- 
ment". If the same document was transmitted before, at 
step S843 the object for filing or re-transmission is ex- 

40 tracted. 

When the date of the document is a fairly old date, 
program control moves to step S836, whereat a check 
is performed to determine whether or not the same doc- 
ument was transmitted belore. When the document was 

45 transmitted before, program control moves to step 
S843. When the document was not transmitted, pro- 
gram control goes to step S837 whereat a check is per- 
formed to determine whether any other date is de- 
scribed in the information for the document. When there 

so is another date, program control goes to step S843. 
When no other date is found, it is ascertained that the 
object can be transmission of the document, and also 
that the date may be wrong. 

When the date of the document is a date fairly far 

55 in advance, at step S840 a check is performed to deter- 
mine whether any other date is described in the infor- 
mation for the document. When, at step S841 , there is 
another date, the object is determined to be "transmis- 
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sion of the document". When no other date is found, pro- 
gram control moves to step S839. 

[Twenty-seventh Embodiment] 

Fig. 84 is a diagram showing an example arrange- 
ment of a system according to this embodiment. In this 
embodiment, the system serves as a current facsimile 
machine. 

According to the system in this embodiment, with- 
out a user specifying an addressee, the addressee for 
a document is determined from bar code on a document 
that is read or information on a cover sheet, so that the 
document can be transmitted to a correct addressee. 

As a result, appropriate information can be trans- 
mitted to a printer or a facsimile machine, by E-mail or 
via the WWW. 

[Twenty-eighth Embodiment] 

Fig. 85 is a diagram illustrating an example arrange- 
ment of a system according to this embodiment. In this 
example, the system implements a desk sorter function 
for processing all the kinds of documents on the desk. 

That is, in consonance with the contents of the doc- 
ument that is read, sorting, filing, scheduling, data ex- 
traction and automatic processing are performed. 

[Twenty-ninth Embodiment] 

Fig. 86 is a detailed diagram for explaining the sys- 
tem in Fig. 85. The processing performed by the system 
shown in Fig. 86 will be specifically described while re- 
ferring to the flowchart in Fig. 87. 

Fig. 87 is a flowchart showing the processing forthis 
embodiment. Figs. 88 and 89 are tables for knowledge 
used during the processing; the table in Fig. 88 shows 
the knowledge in general knowledgebase, and the table 
in Fig. 89 shows the knowledge in a knowledge base of 
a field specified on a cover sheet. In this embodiment, 
information defined in the tables in Figs. 88 and 89 is 
employed to compare a character string included in an 
input document with a character string defined in a char- 
acter string column in the table, and the conception in a 
corresponding column, a Role for further specifying the 
meaning, and a Condition for specifying succeeding in- 
formation, or an instructed process Action are acquired, 
and analysis tor them is performed. 

In Fig. 87, at step S870 a cover page is scanned 
and an OCR process is performed. At step S871 the 
knowledge base in Fig. 88 is examined to extract from 
it data, such as the names of a sender and a receiver 
and their telephone numbers and facsimile numbers. At 
step S872 the obtained data are registered in a data- 
base. In the example in Fig. 86, character string "To: 
Macrohard Corp." is included in the cover page of a re- 
ceived document When this character string is com- 
pared with an item defined in the character string column 



in Fig. 88. Then, matching item "To" is found and its Role 
is a reception company name or an individual name, so 
that "Macrohard Corp." is extracted as information con- 
cerning a receiver. 

s At step S873 a notification method and means, a 

filing job, an action for a place, and a job are extracted. 
In the example in Fig. 86, character string "File: MH/ 
Contract" is included in the cover page of the received 
document. When this character string is compared with 

to the items defined in the character string column in Fig. 
89, matching item "File" is found, filing is extracted from 
the column as an instructed action, and 'MH/Contracf 
is extracted from the document as information for a filing 
place. 

is Assuming that Datel = November 25, 1996 and 
Date2 = November 25, 1996, as is explained in Figs. 
83A and 83B, it is determined that the object is trans- 
mission of a document. 

At step S874 a check is performed to determine 
20 whether an action tor the object of the processing is 
present. Since the transmission of the document to a 
receiver is the object, at step S875 a transmission action 
is performed- At step S876 the information on the cover 
page is employed to determine whether the filing is nec- 
25 essary. Since the filing is required in the example in Fig. 
86, at step S877 the document is filed at the instructed 
place, "MH/Contract". 

At step S878 as well as at step SB76 the information 
on the cover page is employed to determine whether or 
30 not a notification for the transmission of the document 
should be issued to the sender. Since it is instructed to 
notify the sender, by voice, of the transmission of the 
document, at step S879 the sender is notified by voice 
of that the document has been transmitted to the receiv- 
es er. At step S880 whether or not another action is deter- 
mined. At step S881 a keyword is designated, and an 
index is prepared by using keyword ABC. 

[Thirtieth Embodiment] 

40 

Fig. 90 is a diagram showing an example process- 
ing where the history in the past is referred to based on 
an instruction by voice in natural language. The 
processing in Fig. 90 will be explained while referring to 

45 the flowchart in Fig. 91. 

Fig. 91 is a flowchart showing the processing for this 
embodiment. When an aural instruction "Fax Contract 
again to John. Notily him by Phone" is received from a 
user, the system analyzes the object of the instruction 

50 in the natural language, and understands "Re-transmit 
document ■Contract'' to John, and Notify him of the 
transmission". Thus, the processing in Fig. 91 is initiated 
to specify the document Contract and the addressee 
John. 

ss Since re-transmission means that the document 

was previously transmitted, at step S910 history infor- 
mation is acquired from the database. Then, the docu- 
ment Contract and the addressee John are specified 
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from the history information. At step S911 the address 
of the John is obtained from the personal database. 
Since, in the database, there are two Johns, John Smith 
and John Bush, ordinarily the addressee can not be 
specified. Since at step S910 the history information is 
referred to, however, it is judged that John to whom the 
document Contract was transmitted is John Smith, and 
the addressee is thus specified. At step S912 the doc- 
ument "Contract" is acquired from the filing database. 
At step S91 3 the document is transmitted to addressee 
John. Since there is notice instruction "Notify him by 
Phone" at step S914 it is determined that transmission 
of the notice is necessary. At step S915, as well as in 
Fig. 89, according to the rule of the knowledge base, an 
instruction is issued to a facsimile machine 901 as a re- 
ception apparatus to send a notice to a receiver by 
phone. At step S91 6 the other actions are not instructed, 
and program control thereafter determined. 

Upon the receipt of the document, according to the 
notification instruction, the facsimile machine 901 noti- 
fies John by phone of the receipt of the document from 
Mr. Doors. When the facsimile machine 901 can not no- 
tify the addressee by phone of the receipt of the docu- 
ment, the addressee may take its place. 

[Thirty-first Embodiment] 

Fig. 95 is a flowchart showing example processing 
where, from the contents of a document that is input and 
analyzed, it is determined that a user should do some- 
thing, a query is actually transmitted to the user, and a 
required process is preformed automatically. Fig. 92 is 
a diagram showing an example input document. Fig. 93 
is a diagram showing the contents of a conversation be- 
tween a system and a user. Fig. 94 is a diagram showing 
a document that is prepared by the system as the result 
of judgement of the conversation in Fig. 93. 

The processing for preparing an output document 
in Fig. 94 from an input document in Fig. 92 will be de- 
scribed while referring to the flowchart in Fig. 95. 

At step S950 the input document in Fig. 92 is ana- 
lyzed and the following objects are understood: (a) Mr. 
John Smith arrives at Yokohama at three o'clock on No- 
vember 28; (b) Mr. John Smith little understands Japa- 
nese; and (c) some one should meet him at the station. 

At step S951 what the user should do is determined 
from the contents that are obtained. Since it is deter- 
mined that some one must meet him at the station, pro- 
gram control advances to step S952. At step S952 a 
check is performed to determine whether a proxy (sys- 
tem) may execute the action. Since the system can not 
meet him at the station, program control goes to step 
S953. At step S953 a check is performed to determine 
whether an inquiry should be issued to the user, and if 
so, program control moves to step S954. In this exam- 
ple, since it is determined from the addressee of the in- 
put document that this is a request for Dr. Aruna Rohra, 
program control moves to step S954. 



At step S954 a process for inquiring the user (Dr. 
Aruna Rohra) is performed. In this case, as the result 
obtained at step S951 the user must meet John at the 
station, and an inquiry what to do is issued to the user. 

5 At step S955 what to do next is determined from the re- 
sult of the inquiry, and whether or not an action is to be 
initiated is determined. In this example, the user in- 
structs to ask Mr. Tanaka to meet him, the action is ini- 
tiated according to the instruction. Since there is a re- 

io quest for Mr. Tanaka to act as a proxy, it is assumed that 
the action must be initiated. At step S956 a document 
for requesting Mr. Tanaka as a proxy is prepared and a 
contact is made with Mr. Tanaka. In this example, an 
electronic mail document is prepared and transmitted. 

75 in this example, information that John arrives on 28th, 
an instruction tor meeting him for the user and the orig- 
inal document that caused the request are attached to 
automatically form the document. 

As is described above, processing has been ex- 

20 plained for performing the matter (meeting John at the 
station) where from the input document the user must 
actually perform an action. 

[Thirty-second Embodiment] 

2S 

Fig. 96 is a diagram illustrating a thirty-second em- 
bodiment where a system is operated by a controller 

According to the system in this embodiment, devic- 
es and a controller directly or indirectly communicate 
30 with each other to specify the other. 

Therefore, the following functions are provided: (1) 
automatic specifying of a device; (2) acquisition of a de- 
vice capability from the device; (3) a Ul (User Interface) 
inherent to a device; (4) operation by voice; (5) common 
35 basic operation for all the devices; (6) a Ul inherent to 
a user; and (7) a Ul having a high degree of freedom in 
consonance with the status. 

That is, (1 ) only when a controller faces an operat- 
ing target device, it automatically recognizes the target 
40 device; (2) information for each device is acquired from 
the pertinent device via IrDA or via a wireless LAN; (3) 
a Ul inherent to each device is provided from the infor- 
mation for the device; (4) an operation by voice can be 
performed; (5) common basic operation can be provided 
45 by using the same controller; (6) user identification in- 
formation is employed to acquire necessary information 
for each user from the database or the address book 
and to implement the Ui inherent to the user; and (7) the 
optimal Ul is provided in consonance with the status. 
so in Fig. 96 since a controller 960 faces a facsimile 

machine 963, it acquires model identification informa- 
tion from the facsimile machine 963, and renders a Ul, 
which corresponds to the model, active. Then, following 
the displayed "Fax to* character string "Fax to John" is 
55 input with a touch pen, and transmitted to the facsimile 
machine 963. John's facsimile number is read from an 
address book 965 in a personal computer 964, and a 
document set in the facsimile machine 963 or a docu- 
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ment designated on a file in the personal computer 964 
is transmitted to the fax number. The detailed process- 
ing will be described later. 

[Thirty-third Embodiment] 

Fig. 97 is a diagram showing a status monitor. 

In a system according to this embodiment, informa- 
tion of an apparatus that is other than the apparatus that 
a user directly operates can be referred to. 

Therefore, the following functions can be provided: 
(1) remote and handy management; (2) specifying of au- 
tomatic device; (3) common input/output for examining 
a status; (4) the same function as a portable telephone; 
(5) operation by voice; and (6) handling of all the devices 
that can input/output infrared rays. 

That is, (1) required information can be acquired 
even from a remote area from a target device; (2) only 
when a controller faces the target device, it can auto- 
matically recognize the target device; (3) common basic 
operation can be provided by using the same controller; 
(4) the usability is enhanced by providing the portable 
telephone function for the controller; (5) operation by 
voice can be performed; and (6) the common protocol 
is employed to cope with all the devices that can input 
and output infrared rays. 

In Fig. 97, a status monitor 970 selects a printer 961 , 
a copter 962 or a facsimile machine 963, and character 
string "Status 11 is input with a touch pen and instructed. 
The status monitor 970 retrieves status information from 
the device that the status monitor 970 faces, so that the 
user of the status monitor 970 can confirm the statuses 
of the devices. Even when a personal computer 964 is 
located at an area where a command from the status 
monitor 970 can not reach, the status of the personal 
computer 964 is requested to a printer 961, which can 
communicate with the personal computer 964 via the 
network. Then, the status information can be retrieved 
to the status monitor 970 from the personal computer 
964 via the network and the printer 961. 

[Thirty-fourth Embodiment] 

Fig. 98 is a diagram showing an example where a 
controller 960 recognizes a target model, and a corre- 
sponding Ul is read from the controller 960 and dis- 
played. The controller 960, which does not face a target 
device, has a default screen on which message "Select 
a device which you would like to use" ts displayed, as is 
shown in the center in Fig. 98. 

The communication between the controller and the 
target model will now be described. It should be noted 
that (2) and (3) are not necessarily required. When, for 
example, in (4) a target model generates a signal to the 
controller at predetermined intervals, the same effect 
can be obtained. The positional relationship between 
the controller 960 and a target model may be employed 
to determine that the controller 960 selects the target 



model. Further, in (4) the controller 960 may identify a 
target model by examining the shape of the model, or 
by reading the bar code attached to the target model. In 
other words, only the essential portion is shown in Fig. 
5 98. 

(1) A user directs the controller 960 to a target mod- 
el. 

(2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits to the target model a target 
model identification signal request signal. Or, the 
controller 960 is set in advance to transmit a target 
model identification signal request signal at a pre- 
determined interval. 

(3) The controller 960 transmits a target model iden- 
tification signal request signal to the target model. 

(4) A target model identification signal is iransmitted 
upon the receipt of the target model identification 
signal request signal from the controller 960, or ac- 
cording to the setup that a target model identifica- 
tion signal request signal at a specified interval. 

(5) The controller 960 that receives the target model 
identification signal determines a Ul in consonance 
with a target model indicated by the signal, and dis- 
plays a corresponding U! internally stored. In this 
case, the identification signal is provided for each 
model; however, when an identification signal is 
provided for each device, a different Ul can be em- 
ployed for each of devices though they are the same 
model. 

Fig. 99 is a flowchart showing the processing per- 
formed by the controller 960 when the controller recog- 
nizes a target model, and reads and displays a corre- 
sponding Ul. The processing performed by the control- 
ler 960 will now be described. 

At step S991 the controller 960 waits until an in- 
struction from a user is input. At step S992 a target mod- 
el identification signal request signal is transmitted by 
an infrared ray communication method in order to obtain 
a target model identification signal from the target mod- 
el. At step S993 the controller 960 waits until the target 
model outputs a target model identification signal. At 
step S994, the Ul corresponding to the received target 
model identification signal is acquired from internal 
memory and is displayed. 

Fig. 100 is a flowchart showing the processing per- 
formed by a target model when the controller 960 rec- 
so ognizes a target model, and reads and displays a cor- 
responding Ul. The processing performed by the target 
model will now be described. 

At step S1001 , the target model waits for a request 
signal from the controller 960. At step S1002 the target 
55 model transmits a target model identification signal by 
an infrared communication method, etc. 
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[Thirty-fifth Embodiment] 

Fig. 101 is a diagram showing an example where a 
controller 950 receives a Ul from a target model and dis- 
plays it. The controller 960, which does not face a target 5 
device, has a default screen, as is shown in the center 
in Fig. 101. 

The communication between the controller and the 
target model will now be described. It should be noted 
that (2) and (3) are not necessarily required. When, for to 
example, in (4) a target model generates a signal to the 
controller at predetermined intervals, the same effect 
can be obtained. The positional relationship between 
the controller 960 and a target model may be employed 
to determine that the controller 960 selects the target 
model. Further, in (4) the controller 960 may identify a 
target model by examining the shape of the model, or 
by reading the bar code attached to the target model. In 
other words, only the essential portion is shown in Fig. 
101. 20 

(1 ) A user directs the controller 960 to a target mod- 
el. 

(2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the ? s 
controller 960 transmits a Ul request signal to the 
target model. Or the controller 960 is set in advance 

to transmit a Ul request signal at a predetermined 
interval. 

(3) The controller 960 transmits a Ul request signal 30 
to the target model. 

(4) A Ul signal is transmitted upon the receipt of the 
Ul request signal from the controller 960, or accord- 
ing to the setup that a Ul request signal at a speci- 
fied interval. 35 

(5) The controller 960 that receives the Ul signal dis- 
plays the Ul. 

Fig. 102 is a flowchart showing the processing per- 
formed by the controller 960 when the controller re- *o 
ceives a Ul from a target model and displays it. The 
processing performed by the controller 960 will now be 
described. 

At step S1021 the controller 960 warts until an in- 
struction from a user is input. At step SI 022 a Ul request 
signal is transmitted by an infrared ray communication 
method in order to obtain a Ul signal from the target 
model. At step S1023 the controller 960 waits until the 
target model outputs a Ul signal, at step S1024, the re- 
ceived Ul is displayed. 50 

Fig. 1 03 is a flowchart showing the processing per- 
formed by a target model when the controller 960 re- 
ceives a Ul from a target model and displays it. The 
processing performed by the target model will now be 
described. 55 

At step S1031 , the target model waits for a request 
signal from the controller 960. At step S1 032 the target 
model transmits a Ul signal by an infrared communica- 



tion method, etc. 
[Thirty-sixth Embodiment] 

Fig. 104 is a diagram showing an example where 
Uls of devices are stored in a wireless LAN server 1041 
and when a controller 960 faces a copier 962 as a target 
model and requests a Ul, a Ul is transmitted, upon a 
request from the copier 962, from the wireless LAN serv- 
er 1041 to the controller 960, and is displayed. 

The controller 960 : which does not face a target de- 
vice, has a default screen, as is shown in the center in 
Fig. 104. 

The communication between the controller and the 
target model will now be described. 

(1 ) A user directs the controller 960 to a target mod- 
el (a copier 962). 

(2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits a Ul request signal to the 
target model. Or, the controller 960 is set in advance 
to transmit a Ul request signal at a predetermined 
interval. 

(3) The controller 960 transmits controller identifi- 
cation data (controller A) to identify itself and a Ul 
request signal to the target model. 

(4) When the target model receives the controller 
identification data and the Ul request signal from the 
controller, the target model transmits, to the server 
1041 that manages the target model, the controller 
identification data (controller A) and target model 
identification data (copier) to identify itself, and re- 
quests the transmission of the Ul. 

(5) The server 1041 transmits the Ul of the target 
model, which is indicated by the target model data, 
via a wireless LAN to the controller 960, which is 
indicated by the controller identification data. 

(6) The controller 960 receives the Ul addressed to 
itself, and displays the Ul. 

Fig. 105 is a flowchart showing the processing per- 
formed by the controller 960 when the controller re- 
ceives a Ul via the wireless LAN, instead of directly re- 
ceiving it from a target model, and displays it. 

The processing performed by the controller 960 will 
now be described. At step S1051 the controller 960 
waits until an instruction from a user is input. At step 
S1052 a Ul request signal for requesting a Ul and con- 
troller identification data for identifying a controller are 
transmitted to the target model by an infrared ray com- 
munication method. At step Si 053 the controller 960 
waits until a Ul signal is output by any device. At step 
S1054, the received Ul is displayed. 

Fig. 106 is a flowchart showing the processing per- 
formed by the target model when the controller receives 
a Ul via the wireless LAN, instead of directly receiving 
it from a target model, and displays it. The processing 
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performed by the controller 960 will now be described. 

At step S1061 the controller 960 waits until an in- 
struction Irom a user is input. At step SI 062 controller 
identification data received from the controller 960 and 
the target model identification data are transmitted to 
the server 1041, and issuance of the Ul is requested. 

Fig. 107 is a flowchart showing the processing per- 
formed by the server 1041 when the controller receives 
a Ul via the wireless LAN, instead of directly receiving 
it from a target model, and displays it. The processing 
performed by the server 1041 will now be described. 

At step S1071 the server 1041 waits until a Ul trans- 
mission request signal is transmitted from the target 
model. At step S1072 the Ul of the model, which is in- 
dicated by the target model identification data, is trans- 
mitted to the controller 960, which is indicated by iden- 
tification data received Irom the target model. 

[Thirty-seventh Embodiment] 

Fig. 108 is a diagram showing an example where 
Uls of devices are stored in a wireless LAN server 1041 
and when a controller 960a or 960b faces a facsimile 
machine 963 as a target model and requests a Ul, a dif- 
ferent Ul for each user is transmitted, upon a request 
from the facsimile machine 963, from the wireless LAN 
server 1041 to the controller 960a or 960b, and is dis- 
played. 

The controller 960a or 960b, which does not face a 
target device, has a default screen, as is shown in the 
center in Fig. 108. 

The communication between the user, the control- 
ler 960a or 960b and the target model will now be de- 
scribed. 

(1) A user directs the controller 960a or 960b to a 
target model. 

(2) The user touches a touch panel on the controller 
960a or 960b in order to determine the timing at 
which the controller 960a or 960b transmits a Ul re- 
quest signal to the target model. Or, the controller 
960a or 960b is set in advance to transmit a Ul re- 
quest signal at a predetermined interval. 

(3) The controller 960a or 960b transmits controller 
identification data (controller A or controller B) to 
identify itself, user identification data (a senior or a 
beginner) for identifying the user, and a Ul request 
signal to the target model. 

(4) When the target model receives, from the con- 
troller 960a or 960b, the controller identification da- 
ta, the user identification data and the Ul request 
signal, the target model transmits, to the server 
1041 that manages the target model, the received 
controller identification data, the user identification 
data and the target model identification data to iden- 
tify itself. 

(5) The server 1041 transmits the Ul of the target 
model, which is indicated by the user identification 



data and the target model data, via a wireless LAN 
to the controller 960a or 960b, which is indicated by 
the controller identification data. In the example in 
Fig. 108, a Ul that is used for multi-address com- 
5 munication, which is difficult for beginners, is trans- 

mitted only to seniors, not to beginners. 
(6) The controller 960a or 960b receives the Ul ad- 
dressed to itself, and displays the Ul. 

io In the above example, the controller transmits the 
level data for a user is transmitted as user identification 
data. However, the controller may transmit information, 
such as the names of users, to identify individual users, 
and the target device or the server may identify the level 

ts of a user from the received identification data. An inde- 
pendent Ul may be prepared tor each user. 

The above example has been employed only for the 
Ul. The contents of urgent mail addressed to the user 
or the schedule to be immediately informed are collect- 
ed ed in, for example, the server, and they may be included 
in the Ul while the user identification data that differs 
among the users is referred to, and the resultant data 
can be transmitted to the controller 

Fig. 109 is a flowchart showing the processing per- 

2S formed by the controller 960 when the controller re- 
ceives a different Ul for each user, via the wireless LAN, 
instead of directly receiving it from a target model, and 
displays it. The processing performed by the controller 
960 will now be described. 

30 AtstepS109l the controller 960a or 960b waits until 

an instruction from a user is input. At step S1092 a Ul 
request signal for requesting a Ul, controller identifica- 
tion data for identifying a controller, the user identifica- 
tion data for identifying the user are transmitted to the 

35 target model by an infrared ray communication method. 
At step S1093 the controller 960a or 960b waits until a 
Ul signal is output by any device. At step S1094, the 
received Ul is displayed. 

Fig. 110 is a flowchart showing the processing per- 

40 formed by the target model when the controller receives 
a different Ul for each user, via the wireless LAN, instead 
of directly receiving it from a target mode!, and displays 
it. The processing performed by the controller 960a or 
960b will now be described. 

45 At step S 1 1 01 the controller 960a or 960b waits until 
an instruction from a user is input. At step S1102 the 
controller identification data received from the controller 
960, the user identification data, and the target model 
identification data are transmitted to the server 1041, 

so and the issuance of the Ul is requested. 

Fig. 111 is a flowchart showing the processing per- 
formed by the server 1041 when the controller receives 
a different Ul for each user via the wireless LAN , instead 
of directly receiving it from a target model, and displays 

55 it. Fig. 112 is a table showing Uls that are determined 
by the target model and user identification data. The 
processing performed by the server 1041 will now be 
. described. 
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At step S1 111 the server 1041 waits until a Ul trans- 
mission request signal is transmitted from the target 
model. At step S1112 in accordance with the definitions 
in Fig. 112, the Ul that corresponds to the target model 
identification data and the user identification data is 
transmitted to the controller 960a or 960b, which is in- 
dicated by identification data received from the target 
model. 

[Thirty-eighth Embodiment] 

Figs. 113 and 114 are diagrams illustrating an ex- 
ample where provided is a function that a model to be 
operated in front does not have. The communication be- 
tween a user, a controller and a target model will now 
be described. 

(1 ) A user faces a controller 960 to a target model, 
and designated a desired function. 

(2) The target model receives a request from the 
controller 960 and transmits it to a server 1041. 

(3) The server 1041 transmits a Ul of a model that 
corresponds to the requested function. 

(4) The controller 960 displays a received Ul. 

(5) The user performs operation according to an in- 
struction of the Ul. In this example, the user sets a 
document to be transmitted to a copier 962 in front, 
and the controller 960 instructs the destination to 
execute the operation. 

(6) The target model handles what it can do and re- 
quests another apparatus to process what it can not 
do. In this case, since the copier 962 reads a doc- 
ument but can not facsimile the document, it trans- 
mits to a facsimile machine 963 the document data 
and the contents of the operation instruction. 

(7) The facsimile machine 963 transmits by fax the 
received document data to the instructed destina- 
tion. 

In the above example, to determine an apparatus 
that the copier 962 asks for the function that the copier 
962 can not perform, each apparatus may have knowl- 
edge for other apparatuses. Or, the apparatuses on the 
network may be queried it they can execute the function, 
and according to the result, the apparatus to be asked 
for may be determined. Or an inquiry may be issued to 
the server 1041 that has knowledge of the models that 
correspond to functions. 

[Thirty -ninth Embodiment] 

Fig. 11 5 is a diagram illustrating an example where, 
when a controller can not directly communicate with a 
target model in front because an obstacle is present in 
route between the target model and the controller, indi- 
rect communication is attempted using another route. 

The communication between the user, the control- 
ler and the target model will now be described. It should 



be noted that (2) and (3) are not necessarily required. 
When, for example, in (4) a target model generates a 
signal to the controller at predetermined intervals, the 
same effect can be obtained. In other words, only the 
5 essential portion is shown in Fig. 115. 

(1 ) A user directs the controller 960 to a target mod- 
el 

(2) The user touches a touch panel on the controller 
io 960 in order to determine the timing at which the 

controller 960 transmits a Ul request signal to the 
target model. Or, the controller 960 is set in advance 
to transmit a Ul request signal at a predetermined 
interval. 

75 (3) The controller 960 transmits a Ul request signal 
to the target model. 

(4) A Ul signal is transmitted upon the receipt of the 
Ul request signal from the controller 960, or accord- 
ing to the setup that a Ul request signal at a speci- 

20 fjed interval. 

(5) The controller 960 that receives the Ul signal dis- 
plays the Ul. 

(6) It a Ul is not transmitted following a predeter- 
mined period of time after the Ul request signal was 

25 transmitted, or if a Ut is not transmitted during a pe- 

riod longer than a predetermined interval, which is 
set for transmission of Uls, another route is em- 
ployed to transmit a U! request signal. In this exam- 
ple, when infrared communication can not be per- 
30 formed, communication through a wireless LAM is 

attempted. 

(7) The Ul acquired across the LAN is displayed. 

Fig. 1 16 is a flowchart showing the processing per- 
35 formed by a controller for an example where, when the 
controller can not directly communicate with a target 
model in front because an obstacle is present in route 
between the target model and the controller, indirect 
communication is attempted using another route. The 
40 processing performed by the controller 960 will now be 
described. 

At step S1161 the controller 960 waits until an in- 
struction from a user is input. At step S1162aUl request 
signal for requesting a Ul and controller identification da- 
45 ta for identifying a controller are transmitted to the target 
model by an infrared ray communication method. At 
step S1 163 a check is performed to determine whether 
or not the Ul has been received. At step S1164 a check 
is performed to determine whether a predetermined pe- 
so riod of time has elapsed following the transmission of 
the request. At step S1165 a Ul request signal, for re- 
questing a Ul, and controller identification data, for iden- 
tifying a controller, are transmitted by a route, such as 
across a wireless LAN, that is different from the route at 
55 step S1162. At step S1166 the received Ul is displayed. 

In this case, the Ul request signal that is directly 
transmitted to the target model is to be sent to the wire- 
- less LAN server 1 041 . However, since the wireless LAN 
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server 1041 that receives the Ul request signal is not 
sure of a target model, first a menu for selecting a model 
(function) is displayed. A Ul corresponding to the select- 
ed model is displayed. Upon receipt of the model selec- 
tion signal from the controller 960, the wireless LAN 
server 1041 may transmit to the controller 960 a Ul that 
corresponds to the model, or may transmit the menu and 
a plurality of Uls available from the menu. 

[Fortieth Embodiment] 

Fig. 117 is a diagram showing an example where 
an operation and an action performed by a user, the time 
and associated important information are stored as his- 
tory. 

An explanation will be given for the processing lor 
storing, as history, the operation/action perlormed by 
the user, the time and the associated important informa- 
tion. 

(1) A user sets a document in a facsimile machine 
963, and instructs the transmission of the document 
and the user's name (sender's name) to Mr. a of A 
company. 

(2) According to the instruction, the facsimile ma- 
chine 963 transmits the document to Mr. a. Also ac- 
cording to the instruction from Tom, the facsimile 
machine 963 notifies the server 1 041 of the fact that 
the document has been transmitted to Mr. a togeth- 
er with the transmitted document and the transmis- 
sion time. 

(3) The server 1041 saves the received document 
as a file "File ABC" and stores the fact of the receipt 
as history information. 

Fig. 118 is a flowchart showing the processing per- 
formed by the controller for an example where an oper- 
ation and an action perlormed by a user, the time and 
the associated important information are stored as his- 
tory data. The processing performed by the controller 
will now be described. 

At step S1181 the controller 960 waits until an in- 
struction from a user is input. At step S1 182 a request 
signal, for the instruction of the user, controller identifi- 
cation data, for identifying a controller, and the user 
identification data, for identifying the user, are transmit- 
ted by an infrared ray communication method. At step 
S1183 the controller 960 waits until a Ul signal is output 
by any device. At step S1184, the received Ul is dis- 
played. 

Fig. 119 is a flowchart showing the processing per- 
formed by the target model for an example where an 
operation and an action performed by a user, the time 
and the associated important information are stored as 
history data. The processing performed by the target 
model will now be described. 

At step S1 1 91 the target model waits until a request 
signal is transmitted by the controller 960. At step S1 1 92 



a process corresponding to the request signal from the 
controller 960 is performed. In the example shown in 
Fig. 117 a document set by the user is read and trans- 
mitted to Mr. a of A company At step S1193 controller 

5 identification data received from the controller 960, the 
.user identification data, and target model identification 
data, and important data associated with the action that 
was performed, are transmitted to the server 1401. In 
the example in Fig. 117, according to the instruction from 

10 Tom, the fact that the document was transmitted to Mr. 
a is sent to the server 1 041 , together with the transmis- 
sion time and the transmitted document that is the im- 
portant information associated with the fact. 

Fig. 120 is a flowchart showing the processing per- 

75 formed by the server for an example where an operation 
and an action perlormed by a user, the time and the as- 
sociated important information are stored as history da- 
ta. Fig. 121 is a diagram showing example history infor- 
mation that is updated as the result of processing. The 

20 processing performed by the server will now be de- 
scribed. 

At step S1201 the server waits until it receives a 
request signal from the target model. At step S1202 the 
important information associated with the received fact 
2S is stored. In the example in Fig. 117 the document the 
transmission of which is instructed by Tom is saved as 
File ABC. 

At step S1203 the history data are updated in con- 
sonance with the received fact. In the example shown 

30 in Fig. 117 the fact that Tom transmitted a document to 
M. a of A company at 10:00 on July 5, 1 996 is recorded 
together with the file name stored at step S1201. Fig. 
121 is a table showing updated history data; the fact that 
Mary printed ten copies of File XYZ on July 3, 1 996, and 

35 the fact that Tom transmitted File ABC to Mr. a of A com- 
pany at 10:00 on July 5, 1996. At step S1204 a Ul that 
indicates the normal termination of the processing is 
transmitted to the controller that is defined by the con- 
troller identification data received from the target model. 

40 

[Forty-first Embodiment] 

Figs. 122 and 123 are diagrams showing an exam- 
ple where an operation/action performed by a user, the 

45 time and associated important information are stored as 
history data and based on the history data an operation 
is performed. 

An explanation will be described the processing for 
the example where an operation/action performed by a 

so user; the time and associated important information are 
stored as history data and based on the history data an 
operation is performed. 

(1) A controller 960 instructs a copier 962 to show 
55 the history of Tom. 

(2) Upon receipt of the instruction, the copier 962 
asks a server 1401 for managing the history to show 
the history of Tom. 
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(3) The server 1041 refers to the history data and 
transmits, as a Ul, only the history associated with 
the Tom to the copier 962. The copier 962 transmits 
the received history data to the controller 960. 

(4) Tom selects a specific fact from the history data 
displayed on the controller 960, and instructs an ac- 
tion. In this case, since Tom needs a material that 
was previously sent to A company, the fact "File 
ABC was sent to Mr a of A company" is selected 
and action "copy" is instructed to acquire the mate- 
rial from the copier 962 in front. 

(5) The copier 962 transmits to the server 1041 the 
instruction "copy File ABC from the user. 

(6) The server 1041 sends a corresponding docu- 
ment "File ABC" to the copier 962. 

(7) The copier 962 prints a document received by 
the server 1041. 

In the above example, the fact "File ABC was sent 
to Mr. a of A company" is selected from the history in 
order to acquire necessary material from the copier 962 
in front. When the name of document "File ABC" to be 
output is obtained, the list of files can be displayed in- 
stead of the history data and the file name can be se- 
lected, or the file name can be input direcHy 

In addition, in the above example, the document to 
be extracted is the one 1hat is transmitted by the facsim- 
ile machine 963 and stored in the server 1041 . A docu- 
ment stored in a PC across the network can also be em- 
ployed. 

Fig. 124 is a diagram showing the structure of the 
controller 960 used in this embodiment. 

A display 1 241 is used to display data on a liquid 
crystal screen. A resistance film is attached to the dis- 
play 1241, and data can be input with a pen 1242. A 
button on a screen may be selected by the pen 1242. 
Furthermore, a character recognition function is provid- 
ed, and the characters written with the pen 1242 can be 
input. Instead of this function, a voice input/recognition 
function may be employed to input information. 

The input information can be transmitted from a 
communication unit 1243 to another apparatus. A CPU 

1 244 for controlling and a storage unit 1 245 are provided 
inside the apparatus, and input information and informa- 
tion received by the communication unit 1243 and the 
history of the operations are stored. The storage unit 

1245 can be a SRAM card, an SIMM or an HDD. The 
controller 960 is battery operated and water resistant. 

Fig. 125 is a diagram illustrating another example 
structure of the controller 960 used in the embodiment. 

A display 1251 is used to display data using liquid 
crystal or a CRT A touch panel can be additionally pro- 
vided as an option to the display 1251 . Thus, information 
can be input by a finger or a pen. An input unit 1252 is 
used to input information by manipulation of a button. 
The input data can be transmitted from a communication 
unit 1253 to another apparatus. A CPU 1254 for control- 
ling and a storage unit 1255 are provided inside the ap- 



paratus, and input information and information received 
by the communication unit 1253 and the history of the 
operations are stored. The storage unit 1255 can be a 
SRAM card, an SIMM or an HDD. 

$ The controller in Fig. 125 can be used as a button 

controller with the storage unit 1 255 being removed. The 
controller can be used as a personal digital assistant 
(PC) with the communication unit 1253 being removed. 
In addition, the controller can be used as is shown in 

10 Fig. 124, with the input unit 1252 being removed and 
with the touch panel provided. 

Fig. 126 is a diagram showing an additional exam- 
ple structure of the controller 960 used in this embodi- 
ment. 

15 A display 1261 is used to display data using liquid 
crystal. An input unit 1262 is used to input information 
by manipulating a button. The input data can be trans- 
mined by an infrared communication unit 1 263 to anoth- 
er apparatus. A CPU 1264 for controlling and a storage 

20 unit 1265 are provided inside the apparatus, and input 
information and information received by the infrared 
communication unit 1 263 and the history of the opera- 
tions are stored. The storage unit 1265 can be a SRAM 
card, an SIMM or an HDD Furthermore, a PCMCIA slot 

25 1266 is provided, and, for example, a PCMCIA card is 
loaded into the slot 1 266 to enable the connection by a 
PCMCIA bus, so that two communication channels can 
be employed. The infrared communication unit 1263 
may be used only for transmission by light emission, and 

30 the PCMCIA bus may be used for reception Of course, 
the connection by the PCMCIA bus may not be em- 
ployed 



35 Claims 

1. An information processing system that includes a 
first device and a second device that is connected 
to said first device, said first device comprising: 

40 

first reception means for receiving information, 
identification means for, as a plurality of proc- 
esses correspond to said information, distin- 
guishing between a first process to be per- 
- 45 formed by said first device and a second proc- 

ess to be performed by said second device, 
first execution means for performing said first 
process, and 

instruction means for instructing said second 
50 device to perform said second process; 

and said second device comprising: 
second reception means for receiving an in- 
struction from said first device, and 
second execution means for performing said 
55 second process. 

2. An information processing system according to 
claim 1 , wherein said second device includes: 
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determination means tor determining, based on 
said instruction, whether or not said second de- 
vice is currently prepared to perform said sec- 
ond process; and 

notification means for, when it is ascertained 
said second process can not be performed, 
transmitting a notification to that effect to said 
first device. 

3. An information processing system according to 
claim 1, further comprising selection means for, 
when a notification that said second process can 
not be performed is received from said notification 
means, selecting a device to perform said second 
process from among devices other than said sec- 
ond device. 

4. An information processing system according to 
claim 3 t wherein said first device is connected to a 
third device and said selection means selects said 
third device. 

5. An information processing system according to 
claim 3, wherein said selection means selects said 
first device. 

6. An information processing system according to 
claim 1, wherein said first device includes query 
means for, upon receipt of a notification that said 
second process can not be performed, asking a us- 
er to designate a device to perform said second 
process. 

7. An information processing system according to 
claim 1 , wherein said first reception means receives 
as information a processing command and an ob- 
ject to be processed. 

8/ An information processing system according to 
claim 1 , wherein said identification means includes 
determination means for analyzing said information 
that is received and for specifying said plurality of 
processes corresponding to said information. 

9. An information processing system according to 
claim 1 , wherein said first reception means receives 
as said information an output command and an ob- 
ject to be output, said first execution means outputs 
said object to be output in accordance with said out- 
put command, and said instruction means instructs 
said device to perform an additional process con- 
cerning said information. 

10. An information processing system according to 
claim 9, wherein said second device is connected 
to a fourth device, and wherein transfer means is 
provided Jor transferring to said fourth device said 
object to be output 



11. An information processing system according to 
claim 1 0, wherein said transfer means changes said 
object to be output into an output form suitable for 
said fourth device, and transfers said object in said 

5 output form. 

12. An information processing system according to 
claim 9, wherein said second device includes anal- 
ysis means for analyzing the contents of said object 

10 to be output, and wherein, as the result of an anal- 
ysis by said analysis means, said transfer means, 
as needed, transfers said object to be output to said 
fourth device. 

75 13. An information processing system according to 
claim 1 2, wherein said second device includes iden- 
tification means tor identifying the status of a recip- 
ient, and wherein said second device initiates an in- 
formation transfer to said fourth device when said 

20 identification means ascertains that the results will 

not be immediately collected by a recipient even 
when said second device outputs the results to said 
recipient. 

25 14. An information processing system according to 
claim 9, wherein said additional process is a proc- 
ess for sending a notification that an output process 
has been performed. 

30 15. An information processing system according to 
claim 14, wherein voice is used by said second ex- 
ecution means to issue a notification. 

16. An information processing system according to 
35 claim 14, wherein said analysis means identifies a 
recipient and said second execution means issues 
a notification to said recipient by telephone, by us- 
ing a pager or by sending electronic mail. 

40 17. An information processing system according to 
claim 1, wherein, when said first reception means 
receives a request for an instruction to perform a 
specific process from said second device, said 
identification means identifies said specific process 

45 as said second process. 

18. An information processing method, for an informa- 
tion processing system that includes a first device 
and a second device that is connected to said first 
50 device, comprising: 

a reception step of receiving information at said 
first device, 

an identification step of, among a plurality of 
55 processes correspond to said information is re- 

ceived at said first device, distinguishing be- 
tween a first process to be performed by said 
first device and a second process to be per- 
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formed by said second device, 
a first execution step of performing said first 
process at said first device, and 
an instruction step of said first device transmit- 
ting an instruction to perform said second proc- 
ess to said second device; and 
a second execution step of said second device 
performing said second process upon receipt 
of said instruction from said first device. 

19. An information processing method according to 
claim 18, wherein said second device performs: 

a determination step of determining, based on 
said instruction, whether or not said second de- 
vice is currently prepared to perform said sec- 
ond process; and 

a notification step of, when it is ascertained said 
second process can not be performed, trans- 
mitting a notification to that effect to said first 
device. 

20. An information processing method according to 
claim 18, further comprising a selection step of, 
when a notification that said second process can 
not be performed is received, selecting a device to 
perform said second process from among devices 
other than said second device. 

21. An information processing method according to 
claim 20, wherein said first device is connected to 
a third device and wherein said third device is se- 
lected at said selection step. 

22. An information processing method according to 
claim 20, wherein said first device is selected at said 
selection step. 

23. An information processing method according to 
claim 16, wherein said first device performs a query 
step of, upon receipt of a notification that said sec- 
ond process can not be performed, asking a user 
to designate a device to perform said second proc- 
ess. 

24. An information processing method according to 
claim 18, wherein a processing command and an 
object to be processed are received as information 
at said first reception step. 

25. An information processing method according to 
claim 18, wherein said identification step includes a 
determination step of analyzing said information 
that is received and of specifying said plurality of 
processes corresponding to said information. 

26. An information processing method according to 
claim 18, wherein an output command and an object 



to be output are received as said information at said 
first reception step, wherein at said first execution 
step, said object is output in accordance with said 
output command, and wherein, at said instruction 
5 step, said device is instructed to perform an addi- 

tional process concerning said information 

27. An information processing method according to 
claim 26, wherein said second device is connected 

10 to a fourth device, and wherein a transfer step is 
included for transferring to said fourth device said 
object to be output. 

28. An information processing method according to 
is claim 27, wherein at said transfer step, said object 

to be output is changed into an output form suitable 
for said tounh device, and said object in said output 
form is transferred. 

20 29. An information processing method according to 
claim 26, wherein said second device performs an 
analysis step of analyzing the contents of said ob- 
ject to be output, and wherein, as the result of an 
analysis at said analysis step, said transfer step is 

2S performed, as needed. 

30. An information processing method according to 
claim 29, further comprising: 

30 an identification step of said second device 

identifying the status of a recipient, 
wherein said transfer step is performed when it 
is ascertains at said identification step that the 
results will not be immediately collected by a 

35 recipient even when said second device out- 

puts the results to said recipient. 

31. An information processing method according to 
claim 26, wherein said additional process is a proc- 

40 ess for sending a notification that an output process 
has been performed. 

32. An information processing method according to 
claim 31 , wherein voice is used at said second ex- 

45 ecution step to issue a notification. 

33. An information processing method according to 
claim 31, wherein a recipient is identified at said 
analysis step, and wherein, at said second execu- 

50 tion step, a notification is issued to said recipient by 
telephone, by using a pager or by sending electron- 
ic mail. 

34. An information processing method according to 
•55 claim 1 8, wherein, when a request for an instruction 

to peiiorm a specific process from said second de- 
vice is received at said first reception step, said spe- 
cific process is identified as said second process at 
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said identification step. 

35. An information processing apparatus for connec- 
tion to another information processing apparatus, 
comprising: 5 

reception means for receiving information; 
identification means for, among a plurality of 
processes correspond to said information, dis- 
tinguishing between a first process to be per- 10 
formed by said information processing appara- 
tus and a second process to be performed by 
said other information processing apparatus; 
execution means for performing said first proc- 
ess; and 75 
instruction means for instructing said other in- 
formation processing apparatus to perform said 
second process. 

36. An information processing method, for an informa- 20 
tion processing apparatus connected to another in- 
formation processing apparatus, comprising: 

a reception step of receiving information; 
an identification step of, among a plurality of 2S 
processes that correspond to said information, 
distinguishing a first process to be performed 
by said information processing apparatus and 
a second process to be performed by said other 
information processing apparatus; 30 
an execution step of performing said first proc- 
ess; and 

an instruction step ol instructing said other in- 
formation processing apparatus to perform said 
second process. 35 

37. A computer-readable storage medium on which is 
stored an information processing program for con- 
trolling a computer to process information, said pro- 
gram comprising codes for causing the computer to *o 
perform: 

a reception step of receiving information; 
an identification step of, among a plurality of 
processes that correspond to said information, «s 
distinguishing between a first process to be per- 
formed by said information processing appara- 
tus and a second process to be performed by 
said another information processing apparatus; 
an execution step of performing said first proc- so 
ess; and 

an instruction step of instructing said other in- 
formation processing apparatus to perform said 
second process. 

55 
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FIG. 70 
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CANON 

DATE: NOV 25, 1996 

TO: MACROHARD CORP. 
ATTN: MR. JOHN SMITH 
TEL: 001-1 -(41 5)-394-1 1 1 1 
FAX: 001-1-(415)-394-2312 

FROM: CANON INC. 
SENDER: DR. ROHRA 
TEL: +81-44-543-3333 
FAX: +81-44-543-5555 



DEAR SIR, 

THANK YOU FOR YOUR PHONE CALL THIS MORNING. 

PLEASE FIND ENCLOSED HERE WITH THE INFORMATION REGARDING THE 
HOTEL ACCOMMODATION AND OTHER TRAVEL INFORMATION FOR YOUR 
FORTH COMING TRIP TO JAPAN. I WILL BE COMING TO YOKOHAMA CITY AIR 
TERMINAL TO PICK YOU UP, HENCE KINDLY CALL ME AT MY RESIDENCE, 
AFTER YOU BOARD THE LIMOUSINE BUS. 

HOTEL ACCOMMODATION FOR YOUR TRIP IS AS FOLLOWS: 
HOTEL SHIN-YOKOHAMA PRINCE HOTEL 
PERIOD OF STAY: 18TH DEC 1996 - 22ND DEC 1996 (4 NIGHTS) 



HAVE A PLEASANT JOURNEY, 
WITH REGARDS, 



(ARUNA ROHRA) 
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DATE: NOV 22, 1996 




TO: CANON INC. 
ATTN: DR. ARUNA ROHRA 
TEL: +81-44-543-3333 
FAX: +81-44-543-5555 


FROM: MACROHARD CORP. 
SENDER: MR. JOHN SMITH 
TEL: (415) 394-1111 
FAX: (415) 394-2312 


DEAR DR. ROHRA, 




THANK YOU FOR YOUR FAX. 




1 WILL BE REACHING YOKOHAMA AT 3:00 PM ON 28TH NOVEMBER. 
SINCE 1 AM COMING FOR THE FIRST TIME TO YOKOHAMA AND AS 1 AM NOT VERY 
FLUENT IN JAPANESE, 1 WOULD BE GRATEFUL IF YOU COULD KINDLY MEET ME AT 
THE STATION. 


THANKING YOU, 
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